Federico Leva (Nemo) nemowiki@gmail.com writes:

Heather Ford, 01/07/2014 14:37:

...

We want to encourage more research on Wikipedia, not attack the motivations of people we know little about

I'm not sure about the specific wording, but I think the intention is only to stress the importance of open access/open source/open data in research on Wikimedia projects and wikis. I think it's fair for a Wikimedia community publication to call "disappointing" a closed source, non-replicable experiment.

Why should we infer that the experiment is not replicable? Open source software isn't a requirement for that. At the same time, given that the review finds the comparison to Wikipedia "misleading", why should we assume that open source tools would convey any benefit to their putative Wikipedian users? If however the research is sufficiently interesting that the reviewer would like to have a look at the software, wouldn't it be polite to ask the person for a copy, rather than give them a negative review because they (for whatever reasons) didn't already release the software? Maybe they had a good reason for that - if not, they might be happy to share if asked.

From the review:

this reviewer was unable to locate any proof that the developer engaged the Wikipedia community

It's possible that they might have had a good reason for that, too. It probably doesn't make their research less valid *as research* although it would make some claims weaker -- and/or call for further research, as studies usually do. Again, if it's of genuine interest, the interested party just might have to do the good old fashioned thing and ring up the author, thereby rendering them *part* of a community, rather than alienating them since they didn't happen to call first.

+1 Thanks for your really thoughtful comments, Joe, Han-Teng.

Nemo, would you be willing to add a note to the review and/or contacting the researcher?

Best, Heather.

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 05:17, h hanteng@gmail.com wrote:

The tone of the sentence in question

'it is disappointing that the main purpose appears to be completing a

thesis, with little thought to actually improving Wikipedia'

could have been written as

'It would be more useful for the Wikipedia community of practice if

the author discussed or even spelled out the implications of the research for improving Wikipedia".

This suggestion is based on my own impression that [Wiki-research-l]

has mainly two groups of readers: community of practice and community of knowledge. It is okay to have some group tensions for creative/critical inputs. Still, a neutral tone is better for assessment, and an encouraging tone might work a bit better to encourage others to fill the *gaps* (both practice and knowledge ones).

Also, the factors such as originally intended audience and word limits

may determine how much a writer can do for *due weight* (similar to [[WP:due]]). If the original (academic) author failed to address the implications for practices satisfactory, a research newsletter contributor can point out what s/he thinks the potential/actual implications are. (My thanks to the research newsletter's voluntary contributors for their unpaid work!)

While I understand that the monthly research newsletter has its own

perspective and interests different from academic newsletters, it does not sacrifice the integrity of the newsletter to be gentle and specific. I would recommend a minor edit to the sentence as the the newsletter could be read by any one in the world, not just the Wikipedians. It is public/published for all readers, and thus please do not assume the readers know the context of Wikipedia research.

Best,

han-teng liao

2014-07-01 19:37 GMT+07:00 Heather Ford hfordsa@gmail.com:

...

Thanks so much for the newsletter [1]! Always a great read...

But have to just say that comments like this: 'it is disappointing that the main purpose appears to be completing a thesis, with little thought to actually improving Wikipedia' [2] are really harsh and a little unfair. The student is studying Wikipedia - they can hardly only be interested in completing their thesis. We need to remember that researchers are at very different stages of their careers, they have very different motivations, and different levels of engagement with the Wikipedia community, but that *all* research on Wikipedia contributes to our understanding (even if as a catalyst for improvements). We want to encourage more research on Wikipedia, not attack the motivations of people we know little about - particularly when they're just students and particularly when this newsletter is on housed on Wikimedia Foundation's domain.

Best, Heather.

[1] https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June [2] https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June#.22Recommendin...

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk/ Doctoral Programme EthnographyMatters http://ethnographymatters.net/ | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Apologies. You're right, Han-Teng. The reviewer looks to be Piotr Konieczny who I think is on this mailing list?

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 12:58, h hanteng@gmail.com wrote:

Heather, I am not sure who contribute that. Probably not Nemo. If this issue of newsletter is correctly attributed, the contributors include: Taha Yasseri, Maximilian Klein, Piotr Konieczny, Kim Osman, and Tilman Bayer. My suggestion is only a personal one, and I am not sure if it is against policies to make a few edits once the newsletter is out.

Thanks again to the contributors of the newsletter, my life is a bit easier and more interesting because of your work.

2014-07-02 15:35 GMT+07:00 Heather Ford hfordsa@gmail.com:

+1 Thanks for your really thoughtful comments, Joe, Han-Teng.

...

Nemo, would you be willing to add a note to the review and/or contacting the researcher?

Best, Heather.

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 05:17, h hanteng@gmail.com wrote:

...

The tone of the sentence in question

'it is disappointing that the main purpose appears to be completing

a thesis, with little thought to actually improving Wikipedia'

could have been written as

'It would be more useful for the Wikipedia community of practice if

the author discussed or even spelled out the implications of the research for improving Wikipedia".

This suggestion is based on my own impression that [Wiki-research-l]

has mainly two groups of readers: community of practice and community of knowledge. It is okay to have some group tensions for creative/critical inputs. Still, a neutral tone is better for assessment, and an encouraging tone might work a bit better to encourage others to fill the *gaps* (both practice and knowledge ones).

Also, the factors such as originally intended audience and word

limits may determine how much a writer can do for *due weight* (similar to [[WP:due]]). If the original (academic) author failed to address the implications for practices satisfactory, a research newsletter contributor can point out what s/he thinks the potential/actual implications are. (My thanks to the research newsletter's voluntary contributors for their unpaid work!)

While I understand that the monthly research newsletter has its own

perspective and interests different from academic newsletters, it does not sacrifice the integrity of the newsletter to be gentle and specific. I would recommend a minor edit to the sentence as the the newsletter could be read by any one in the world, not just the Wikipedians. It is public/published for all readers, and thus please do not assume the readers know the context of Wikipedia research.

Best,

han-teng liao

2014-07-01 19:37 GMT+07:00 Heather Ford hfordsa@gmail.com:

...

Thanks so much for the newsletter [1]! Always a great read...

But have to just say that comments like this: 'it is disappointing that the main purpose appears to be completing a thesis, with little thought to actually improving Wikipedia' [2] are really harsh and a little unfair. The student is studying Wikipedia - they can hardly only be interested in completing their thesis. We need to remember that researchers are at very different stages of their careers, they have very different motivations, and different levels of engagement with the Wikipedia community, but that *all* research on Wikipedia contributes to our understanding (even if as a catalyst for improvements). We want to encourage more research on Wikipedia, not attack the motivations of people we know little about - particularly when they're just students and particularly when this newsletter is on housed on Wikimedia Foundation's domain.

Best, Heather.

[1] https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June [2] https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June#.22Recommendin...

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk/ Doctoral Programme EthnographyMatters http://ethnographymatters.net/ | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Is wiki research (including Wikipedia research) research on

computer-supported collaborative work (CSCW).

Sure. A lot of it. If it's research of the nature of wikis, then certainly. A lot of papers that study Wikipedia/wikis are published via ACM CSCW[1].

If it's research that just so happens to use Wikipedia as a dataset, then I don't know if we should even call it Wikipedia research. Some research of the content and work patterns in Wikipedia is a mixture of the two. For example, an analysis of the content in Wikipedia might be useful for identifying bias and missing information -- even if that was not original intention of the study and identifying bias and missing information is purely used to tune some content algorithm.

1. http://cscw.acm.org

On Wed, Jul 2, 2014 at 9:15 AM, Finn Årup Nielsen fn@imm.dtu.dk wrote:

Can I ask a silly question: Is wiki research (including Wikipedia research) research on computer-supported collaborative work (CSCW).

My immediate thought was: yes of course. I do note that on Wikipedia there is only a parenthetic mentioning of Wikipedia in "Computer-supported cooperative work" and no mentioning of CSCW in "Wiki" (but I have also heard that you shouldn't trust Wikipedia because anyone can edit).

I suppose that some wiki research could be non-CSCW research? E.g., research on named entity extraction using Wikipedia would not be called CSCW.

best regards Finn Årup Nielsen http://www.compute.dtu.dk/~faan/

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

As computing goes “collaborative and social” [1], the 2012 ACM Computing Classification System (CCS) [2] has made a few modifications that put CSCW under the umbrella called "Collaborative and social computing", along with other social media, blogs, wikis, etc.

Best, han-teng liao

[1]: http://people.oii.ox.ac.uk/hanteng/2012/10/01/computing-goes-collaborative-a... [2]: http://www.acm.org/about/class/2012 [3]: http://dl.acm.org/ccs.cfm

2014-07-02 22:38 GMT+07:00 Carlos Castillo chato@chato.cl:

Hi,

I will go as far as saying that Wikipedia research actually changed the nature of the CSCW field and the CSCW conference. A few years ago an avalanche of Wiki-related papers arrived to the CSCW conference, which (in my opinion as a semi-outsider to that community) added social media as a major element. Indeed in recent years the conference changed name and now is "Computer-Supported Collaborative Work and Social Computing".

Cheers,

-- ChaTo (Carlos Castillo) http://chato.cl/ - https://linkedin.com/in/chato - https://twitter.com/chatox

On Wed, Jul 2, 2014, at 07:15 AM, Finn Årup Nielsen wrote:

...

Can I ask a silly question: Is wiki research (including Wikipedia research) research on computer-supported collaborative work (CSCW).

My immediate thought was: yes of course. I do note that on Wikipedia there is only a parenthetic mentioning of Wikipedia in "Computer-supported cooperative work" and no mentioning of CSCW in "Wiki" (but I have also heard that you shouldn't trust Wikipedia because anyone can edit).

I suppose that some wiki research could be non-CSCW research? E.g., research on named entity extraction using Wikipedia would not be called CSCW.

best regards Finn Årup Nielsen http://www.compute.dtu.dk/~faan/

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

I really like the idea of some kind of annual award.

On 2 July 2014 10:15, Aaron Halfaker aaron.halfaker@gmail.com wrote:

Given that it seems we agree with Poitr's desire for research about Wikipedia to lead to useful tools an insights that can be directly applied to making Wikipedia and other wikis better, what might be a more effective strategy for encouraging researchers to engage with us or at least release their work in forms that we can more easily work with?

Here's a couple of half-baked ideas:

• *Wiki research impact task force* -- contacts authors to encourage

them to release code/datasets/etc. and praise them publicly when they do -- could be part of the work of newsletter reviewers. There are many researchers on this list who work directly with Wikimedians to make sure that their research has direct impact and their awesomeness is worth our appreciation and public recognition.

• *Yearly research award* -- for the most directly impactful research

projects/researchers similar to https://meta.wikimedia.org/wiki/Research:Wikimedia_France_Research_Award. One of the focuses of the judging could be the direct impact that the work has had.

-Aaron

On Wed, Jul 2, 2014 at 7:05 AM, Heather Ford hfordsa@gmail.com wrote:

...

Apologies. You're right, Han-Teng. The reviewer looks to be Piotr Konieczny who I think is on this mailing list?

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 12:58, h hanteng@gmail.com wrote:

...

Heather, I am not sure who contribute that. Probably not Nemo. If this issue of newsletter is correctly attributed, the contributors include: Taha Yasseri, Maximilian Klein, Piotr Konieczny, Kim Osman, and Tilman Bayer. My suggestion is only a personal one, and I am not sure if it is against policies to make a few edits once the newsletter is out.

Thanks again to the contributors of the newsletter, my life is a bit easier and more interesting because of your work.

2014-07-02 15:35 GMT+07:00 Heather Ford hfordsa@gmail.com:

+1 Thanks for your really thoughtful comments, Joe, Han-Teng.

...

Nemo, would you be willing to add a note to the review and/or contacting the researcher?

Best, Heather.

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 05:17, h hanteng@gmail.com wrote:

...

The tone of the sentence in question

'it is disappointing that the main purpose appears to be

completing a thesis, with little thought to actually improving Wikipedia'

could have been written as

'It would be more useful for the Wikipedia community of practice

if the author discussed or even spelled out the implications of the research for improving Wikipedia".

This suggestion is based on my own impression that [Wiki-research-l]

has mainly two groups of readers: community of practice and community of knowledge. It is okay to have some group tensions for creative/critical inputs. Still, a neutral tone is better for assessment, and an encouraging tone might work a bit better to encourage others to fill the *gaps* (both practice and knowledge ones).

Also, the factors such as originally intended audience and word

limits may determine how much a writer can do for *due weight* (similar to [[WP:due]]). If the original (academic) author failed to address the implications for practices satisfactory, a research newsletter contributor can point out what s/he thinks the potential/actual implications are. (My thanks to the research newsletter's voluntary contributors for their unpaid work!)

While I understand that the monthly research newsletter has its

own perspective and interests different from academic newsletters, it does not sacrifice the integrity of the newsletter to be gentle and specific. I would recommend a minor edit to the sentence as the the newsletter could be read by any one in the world, not just the Wikipedians. It is public/published for all readers, and thus please do not assume the readers know the context of Wikipedia research.

Best,

han-teng liao

2014-07-01 19:37 GMT+07:00 Heather Ford hfordsa@gmail.com:

...

Thanks so much for the newsletter [1]! Always a great read...

But have to just say that comments like this: 'it is disappointing that the main purpose appears to be completing a thesis, with little thought to actually improving Wikipedia' [2] are really harsh and a little unfair. The student is studying Wikipedia - they can hardly only be interested in completing their thesis. We need to remember that researchers are at very different stages of their careers, they have very different motivations, and different levels of engagement with the Wikipedia community, but that *all* research on Wikipedia contributes to our understanding (even if as a catalyst for improvements). We want to encourage more research on Wikipedia, not attack the motivations of people we know little about - particularly when they're just students and particularly when this newsletter is on housed on Wikimedia Foundation's domain.

Best, Heather.

[1] https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June [2] https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June#.22Recommendin...

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk/ Doctoral Programme EthnographyMatters http://ethnographymatters.net/ | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115

http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

I feel like that might be a bit short-notice - papers need to be submitted, reviewed or voted on, so on and so forth. But it could be lovely to have a 'best presentation' award for WM itself!

On 2 July 2014 10:33, Edward Saperia ed@wikimanialondon.org wrote:

I really like the idea of some kind of annual award.

...

If someone puts it together before Wikimania, I can put it into the closing ceremony?

*Edward Saperia* Conference Director Wikimania London http://www.wikimanialondon.org/ email ed@wikimanialondon.org • facebook http://www.facebook.com/edsaperia • twitter http://www.twitter.com/edsaperia • 07796955572 133-135 Bethnal Green Road, E2 7DG

...

On 2 July 2014 10:15, Aaron Halfaker aaron.halfaker@gmail.com wrote:

...

Given that it seems we agree with Poitr's desire for research about Wikipedia to lead to useful tools an insights that can be directly applied to making Wikipedia and other wikis better, what might be a more effective strategy for encouraging researchers to engage with us or at least release their work in forms that we can more easily work with?

Here's a couple of half-baked ideas:

• *Wiki research impact task force* -- contacts authors to encourage

them to release code/datasets/etc. and praise them publicly when they do -- could be part of the work of newsletter reviewers. There are many researchers on this list who work directly with Wikimedians to make sure that their research has direct impact and their awesomeness is worth our appreciation and public recognition.

• *Yearly research award* -- for the most directly impactful

research projects/researchers similar to https://meta.wikimedia.org/wiki/Research:Wikimedia_France_Research_Award. One of the focuses of the judging could be the direct impact that the work has had.

-Aaron

On Wed, Jul 2, 2014 at 7:05 AM, Heather Ford hfordsa@gmail.com wrote:

...

Apologies. You're right, Han-Teng. The reviewer looks to be Piotr Konieczny who I think is on this mailing list?

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 12:58, h hanteng@gmail.com wrote:

...

Heather, I am not sure who contribute that. Probably not Nemo. If this issue of newsletter is correctly attributed, the contributors include: Taha Yasseri, Maximilian Klein, Piotr Konieczny, Kim Osman, and Tilman Bayer. My suggestion is only a personal one, and I am not sure if it is against policies to make a few edits once the newsletter is out.

Thanks again to the contributors of the newsletter, my life is a bit easier and more interesting because of your work.

2014-07-02 15:35 GMT+07:00 Heather Ford hfordsa@gmail.com:

+1 Thanks for your really thoughtful comments, Joe, Han-Teng.

...

Nemo, would you be willing to add a note to the review and/or contacting the researcher?

Best, Heather.

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115

http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 05:17, h hanteng@gmail.com wrote:

> The tone of the sentence in question > > 'it is disappointing that the main purpose appears to be > completing a thesis, with little thought to actually improving > Wikipedia' > > could have been written as > > 'It would be more useful for the Wikipedia community of practice > if the author discussed or even spelled out the implications of the > research for improving Wikipedia". > > This suggestion is based on my own impression that [Wiki-research-l] > has mainly two groups of readers: community of practice and community of > knowledge. It is okay to have some group tensions for creative/critical > inputs. Still, a neutral tone is better for assessment, and an encouraging > tone might work a bit better to encourage others to fill the *gaps* (both > practice and knowledge ones). > > Also, the factors such as originally intended audience and word > limits may determine how much a writer can do for *due weight* (similar to > [[WP:due]]). If the original (academic) author failed to address the > implications for practices satisfactory, a research newsletter contributor > can point out what s/he thinks the potential/actual implications are. (My > thanks to the research newsletter's voluntary contributors for > their unpaid work!) > > While I understand that the monthly research newsletter has its > own perspective and interests different from academic newsletters, it does > not sacrifice the integrity of the newsletter to be gentle and specific. I > would recommend a minor edit to the sentence as the the newsletter could be > read by any one in the world, not just the Wikipedians. It is > public/published for all readers, and thus please do not assume the readers > know the context of Wikipedia research. > > Best, > > han-teng liao > > > 2014-07-01 19:37 GMT+07:00 Heather Ford hfordsa@gmail.com: > >> Thanks so much for the newsletter [1]! Always a great read... >> >> But have to just say that comments like this: 'it is disappointing >> that the main purpose appears to be completing a thesis, with >> little thought to actually improving Wikipedia' [2] are really harsh and a >> little unfair. The student is studying Wikipedia - they can hardly only be >> interested in completing their thesis. We need to remember that researchers >> are at very different stages of their careers, they have very different >> motivations, and different levels of engagement with the Wikipedia >> community, but that *all* research on Wikipedia contributes to our >> understanding (even if as a catalyst for improvements). We want to >> encourage more research on Wikipedia, not attack the motivations of people >> we know little about - particularly when they're just students and >> particularly when this newsletter is on housed on Wikimedia Foundation's >> domain. >> >> Best, >> Heather. >> >> [1] https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June >> [2] >> https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June#.22Recommendin... >> >> Heather Ford >> Oxford Internet Institute http://www.oii.ox.ac.uk/ Doctoral >> Programme >> EthnographyMatters http://ethnographymatters.net/ | Oxford >> Digital Ethnography Group >> http://www.oii.ox.ac.uk/research/projects/?id=115 >> http://hblog.org | @hfordsa http://www.twitter.com/hfordsa >> >> >> _______________________________________________ >> Wiki-research-l mailing list >> Wiki-research-l@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >> >> > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > >

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

-- Oliver Keyes Research Analyst Wikimedia Foundation

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

On Wed, Jul 2, 2014 at 7:39 AM, Edward Saperia ed@wikimanialondon.org wrote:

On 2 July 2014 15:37, Oliver Keyes okeyes@wikimedia.org wrote:

...

I feel like that might be a bit short-notice - papers need to be submitted, reviewed or voted on, so on and so forth. But it could be lovely to have a 'best presentation' award for WM itself!

Well, we could pick from things featured in the research newsletter, for example? How do you imagine the winner to be chosen? We can always do something more structured for next year. But this might be a good way to launch the idea of a research award.

Ed

Not an award, but it seems worth mentioning https://wikimania2014.wikimedia.org/wiki/Submissions/The_State_of_Wikimedia_... here ...

(Anyone who is going to be in London and has ideas or feedback about the newsletter: don't hesitate to say hi ;)

...

On 2 July 2014 10:33, Edward Saperia ed@wikimanialondon.org wrote:

...

...

I really like the idea of some kind of annual award.

If someone puts it together before Wikimania, I can put it into the closing ceremony?

Edward Saperia Conference Director Wikimania London email • facebook • twitter • 07796955572 133-135 Bethnal Green Road, E2 7DG

...

On 2 July 2014 10:15, Aaron Halfaker aaron.halfaker@gmail.com wrote:

...

Given that it seems we agree with Poitr's desire for research about Wikipedia to lead to useful tools an insights that can be directly applied to making Wikipedia and other wikis better, what might be a more effective strategy for encouraging researchers to engage with us or at least release their work in forms that we can more easily work with?

Here's a couple of half-baked ideas:

Wiki research impact task force -- contacts authors to encourage them to release code/datasets/etc. and praise them publicly when they do -- could be part of the work of newsletter reviewers. There are many researchers on this list who work directly with Wikimedians to make sure that their research has direct impact and their awesomeness is worth our appreciation and public recognition. Yearly research award -- for the most directly impactful research projects/researchers similar to https://meta.wikimedia.org/wiki/Research:Wikimedia_France_Research_Award. One of the focuses of the judging could be the direct impact that the work has had.

-Aaron

On Wed, Jul 2, 2014 at 7:05 AM, Heather Ford hfordsa@gmail.com wrote:

...

Apologies. You're right, Han-Teng. The reviewer looks to be Piotr Konieczny who I think is on this mailing list?

Heather Ford Oxford Internet Institute Doctoral Programme EthnographyMatters | Oxford Digital Ethnography Group http://hblog.org | @hfordsa

On 2 July 2014 12:58, h hanteng@gmail.com wrote: > > Heather, I am not sure who contribute that. Probably not Nemo. If > this issue of newsletter is correctly attributed, the contributors include: > Taha Yasseri, Maximilian Klein, Piotr Konieczny, Kim Osman, and Tilman > Bayer. My suggestion is only a personal one, and I am not sure if it is > against policies to make a few edits once the newsletter is out. > > Thanks again to the contributors of the newsletter, my life is a bit > easier and more interesting because of your work. > > > > 2014-07-02 15:35 GMT+07:00 Heather Ford hfordsa@gmail.com: > >> +1 Thanks for your really thoughtful comments, Joe, Han-Teng. >> >> Nemo, would you be willing to add a note to the review and/or >> contacting the researcher? >> >> Best, >> Heather. >> >> Heather Ford >> Oxford Internet Institute Doctoral Programme >> EthnographyMatters | Oxford Digital Ethnography Group >> http://hblog.org | @hfordsa >> >> >> >> >> On 2 July 2014 05:17, h hanteng@gmail.com wrote: >>> >>> The tone of the sentence in question >>> >>> 'it is disappointing that the main purpose appears to be >>> completing a thesis, with little thought to actually improving Wikipedia' >>> >>> could have been written as >>> >>> 'It would be more useful for the Wikipedia community of >>> practice if the author discussed or even spelled out the implications of the >>> research for improving Wikipedia". >>> >>> This suggestion is based on my own impression that >>> [Wiki-research-l] has mainly two groups of readers: community of practice >>> and community of knowledge. It is okay to have some group tensions for >>> creative/critical inputs. Still, a neutral tone is better for assessment, >>> and an encouraging tone might work a bit better to encourage others to fill >>> the *gaps* (both practice and knowledge ones). >>> >>> Also, the factors such as originally intended audience and word >>> limits may determine how much a writer can do for *due weight* (similar to >>> [[WP:due]]). If the original (academic) author failed to address the >>> implications for practices satisfactory, a research newsletter contributor >>> can point out what s/he thinks the potential/actual implications are. (My >>> thanks to the research newsletter's voluntary contributors for their unpaid >>> work!) >>> >>> While I understand that the monthly research newsletter has its >>> own perspective and interests different from academic newsletters, it does >>> not sacrifice the integrity of the newsletter to be gentle and specific. I >>> would recommend a minor edit to the sentence as the the newsletter could be >>> read by any one in the world, not just the Wikipedians. It is >>> public/published for all readers, and thus please do not assume the readers >>> know the context of Wikipedia research. >>> >>> Best, >>> >>> han-teng liao >>> >>> >>> 2014-07-01 19:37 GMT+07:00 Heather Ford hfordsa@gmail.com: >>>> >>>> Thanks so much for the newsletter [1]! Always a great read... >>>> >>>> But have to just say that comments like this: 'it is disappointing >>>> that the main purpose appears to be completing a thesis, with little thought >>>> to actually improving Wikipedia' [2] are really harsh and a little unfair. >>>> The student is studying Wikipedia - they can hardly only be interested in >>>> completing their thesis. We need to remember that researchers are at very >>>> different stages of their careers, they have very different motivations, and >>>> different levels of engagement with the Wikipedia community, but that *all* >>>> research on Wikipedia contributes to our understanding (even if as a >>>> catalyst for improvements). We want to encourage more research on Wikipedia, >>>> not attack the motivations of people we know little about - particularly >>>> when they're just students and particularly when this newsletter is on >>>> housed on Wikimedia Foundation's domain. >>>> >>>> Best, >>>> Heather. >>>> >>>> [1] https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June >>>> [2] >>>> https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June#.22Recommendin... >>>> >>>> Heather Ford >>>> Oxford Internet Institute Doctoral Programme >>>> EthnographyMatters | Oxford Digital Ethnography Group >>>> http://hblog.org | @hfordsa >>>> >>>> >>>> >>>> _______________________________________________ >>>> Wiki-research-l mailing list >>>> Wiki-research-l@lists.wikimedia.org >>>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >>>> >>> >>> >>> _______________________________________________ >>> Wiki-research-l mailing list >>> Wiki-research-l@lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >>> >> >> >> _______________________________________________ >> Wiki-research-l mailing list >> Wiki-research-l@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >> > > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

-- Oliver Keyes Research Analyst Wikimedia Foundation

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

-- Oliver Keyes Research Analyst Wikimedia Foundation

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Han-teng,

Could you expand on what you are imagining with these two aspects of impact? Also, I'd like to think that impact wouldn't be so narrow as to be based on engagement only. Surely, researchers can produce things that are highly impactful without explicitly "engaging" with the volunteer community.

-Aaron

On Wed, Jul 2, 2014 at 10:32 AM, h hanteng@gmail.com wrote:

I second Aaron's two suggestions, with a slight change of wordings of the first: (1) change "impact" to "public engagement" (potentially new users) or "community engagement" (existing users)

han-teng liao

2014-07-02 21:15 GMT+07:00 Aaron Halfaker aaron.halfaker@gmail.com:

Given that it seems we agree with Poitr's desire for research about

...

Wikipedia to lead to useful tools an insights that can be directly applied to making Wikipedia and other wikis better, what might be a more effective strategy for encouraging researchers to engage with us or at least release their work in forms that we can more easily work with?

Here's a couple of half-baked ideas:

• *Wiki research impact task force* -- contacts authors to encourage

them to release code/datasets/etc. and praise them publicly when they do -- could be part of the work of newsletter reviewers. There are many researchers on this list who work directly with Wikimedians to make sure that their research has direct impact and their awesomeness is worth our appreciation and public recognition.

• *Yearly research award* -- for the most directly impactful research

projects/researchers similar to https://meta.wikimedia.org/wiki/Research:Wikimedia_France_Research_Award. One of the focuses of the judging could be the direct impact that the work has had.

-Aaron

On Wed, Jul 2, 2014 at 7:05 AM, Heather Ford hfordsa@gmail.com wrote:

...

Apologies. You're right, Han-Teng. The reviewer looks to be Piotr Konieczny who I think is on this mailing list?

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 12:58, h hanteng@gmail.com wrote:

...

Heather, I am not sure who contribute that. Probably not Nemo. If this issue of newsletter is correctly attributed, the contributors include: Taha Yasseri, Maximilian Klein, Piotr Konieczny, Kim Osman, and Tilman Bayer. My suggestion is only a personal one, and I am not sure if it is against policies to make a few edits once the newsletter is out.

Thanks again to the contributors of the newsletter, my life is a bit easier and more interesting because of your work.

2014-07-02 15:35 GMT+07:00 Heather Ford hfordsa@gmail.com:

+1 Thanks for your really thoughtful comments, Joe, Han-Teng.

...

Nemo, would you be willing to add a note to the review and/or contacting the researcher?

Best, Heather.

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 05:17, h hanteng@gmail.com wrote:

...

The tone of the sentence in question

'it is disappointing that the main purpose appears to be

completing a thesis, with little thought to actually improving Wikipedia'

could have been written as

'It would be more useful for the Wikipedia community of practice

if the author discussed or even spelled out the implications of the research for improving Wikipedia".

This suggestion is based on my own impression that [Wiki-research-l]

has mainly two groups of readers: community of practice and community of knowledge. It is okay to have some group tensions for creative/critical inputs. Still, a neutral tone is better for assessment, and an encouraging tone might work a bit better to encourage others to fill the *gaps* (both practice and knowledge ones).

Also, the factors such as originally intended audience and word

limits may determine how much a writer can do for *due weight* (similar to [[WP:due]]). If the original (academic) author failed to address the implications for practices satisfactory, a research newsletter contributor can point out what s/he thinks the potential/actual implications are. (My thanks to the research newsletter's voluntary contributors for their unpaid work!)

While I understand that the monthly research newsletter has its

own perspective and interests different from academic newsletters, it does not sacrifice the integrity of the newsletter to be gentle and specific. I would recommend a minor edit to the sentence as the the newsletter could be read by any one in the world, not just the Wikipedians. It is public/published for all readers, and thus please do not assume the readers know the context of Wikipedia research.

Best,

han-teng liao

2014-07-01 19:37 GMT+07:00 Heather Ford hfordsa@gmail.com:

> Thanks so much for the newsletter [1]! Always a great read... > > But have to just say that comments like this: 'it is disappointing > that the main purpose appears to be completing a thesis, with > little thought to actually improving Wikipedia' [2] are really harsh and a > little unfair. The student is studying Wikipedia - they can hardly only be > interested in completing their thesis. We need to remember that researchers > are at very different stages of their careers, they have very different > motivations, and different levels of engagement with the Wikipedia > community, but that *all* research on Wikipedia contributes to our > understanding (even if as a catalyst for improvements). We want to > encourage more research on Wikipedia, not attack the motivations of people > we know little about - particularly when they're just students and > particularly when this newsletter is on housed on Wikimedia Foundation's > domain. > > Best, > Heather. > > [1] https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June > [2] > https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June#.22Recommendin... > > Heather Ford > Oxford Internet Institute http://www.oii.ox.ac.uk/ Doctoral > Programme > EthnographyMatters http://ethnographymatters.net/ | Oxford > Digital Ethnography Group > http://www.oii.ox.ac.uk/research/projects/?id=115 > http://hblog.org | @hfordsa http://www.twitter.com/hfordsa > > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > >

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

Thanks so much for this, Kerry. And thanks, Aaron for (as always) great, productive suggestions.

I think there are two issues that need to be dealt with separately here. The first is about disparaging remarks made about researchers' contributions that kicked off this discussion. One idea that I had when I saw a similar problem earlier this year was to at least have reviewers add their names to reviews so that we are making a clear distinction between the opinion of a single reviewer and the community/organisation as a whole. Some reviewers have added their names to reviews (thank you!) but I think that needs to be a standard for the newsletter. This probably won't solve the problem completely but hopefully reviewers will be more thoughtful about their critique in the future.

The second is to encourage research about Wikipedia that engages with the Wikimedia community. And yes, I, too, think that awards and acknowledgements are great ideas. I'd say that, when evaluating, engagement is even more important than impact because we want to encourage students and researchers at various stages of their careers (many of whom would not win awards for impact) to engage with the community when working on these projects. Of course, this kind of work is necessarily going to have more impact because Wikimedians themselves are going to be a part of it somehow. For this, I definitely agree with some kind of acknowledgement of research done - beyond, perhaps, just one or two star researchers winning a few awards. This can be done together e.g. awards for best papers in different categories but also acknowledgements for work with the community on particular projects as suggested by Kerry.

Best, Heather.

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 3 July 2014 02:56, Kerry Raymond kerry.raymond@gmail.com wrote:

Having had a work role oversighting many university researchers including PHD and other research students, I think many start out with intentions to engage fully with stakeholders and contribute back into the real world in some way, but it's fair to say that deadline pressures tend to force them to focus their energies into the "academically valued" outcomes, e.g. published papers, theses, etc. This is just as true for Wikipedia-related research as for, say, aquaculture. Of course, some never intended to contribute back, but are solely motivated by climbing the greasy pole of academia.

Because data gathering can be a time-consuming or expensive stumbling block in a research plan, organisations that freely publish detailed data (as WMF does) are natural magnets to researchers who can use that data to study various phenomena which may have broader relevance than just Wikipedia or where the Wikipedia data serves as a ground truth for other experiments or as proxy for other unavailable data. For example, you can use Wikipedia to study categorisation or named entity extraction without having real interest in Wikipedia itself.

So I think it is for those who are passionate about Wikipedia itself to see how such research findings may be used to improve Wikipedia. As for releasing source code, it has to recognised that software in research projects is often very quick-and-dirty and probably not designed to be integrated into the MediaWiki code base. Effective solutions to Wikipedia issues often require a mix of technology and change to community process/culture (which is often far harder to get right).

This is not to say they we should not encourage researchers to "give back", but I think we do need to understand that the reasons people don't give back aren't always attributable solely to "bad faith".

In additions to suggestions already made re awards, just having a letter of commendation on WMF letterhead acknowledging the research and its potential to improve Wikipedia would be a useful thing especially for junior researchers seeking to establish themselves; this kind of external validation is helpful to their CVs. This could be sent to any researchers whose research was deemed to have merit with different wording for those who made (according to some appropriately-appointed group) greater or lesser contributions to real Wikipedia impact.

Sent from my iPad

On 3 Jul 2014, at 12:15 am, Aaron Halfaker aaron.halfaker@gmail.com wrote:

Given that it seems we agree with Poitr's desire for research about Wikipedia to lead to useful tools an insights that can be directly applied to making Wikipedia and other wikis better, what might be a more effective strategy for encouraging researchers to engage with us or at least release their work in forms that we can more easily work with?

Here's a couple of half-baked ideas:

• *Wiki research impact task force* -- contacts authors to encourage

them to release code/datasets/etc. and praise them publicly when they do -- could be part of the work of newsletter reviewers. There are many researchers on this list who work directly with Wikimedians to make sure that their research has direct impact and their awesomeness is worth our appreciation and public recognition.

• *Yearly research award* -- for the most directly impactful research

projects/researchers similar to https://meta.wikimedia.org/wiki/Research:Wikimedia_France_Research_Award. One of the focuses of the judging could be the direct impact that the work has had.

-Aaron

On Wed, Jul 2, 2014 at 7:05 AM, Heather Ford hfordsa@gmail.com wrote:

...

Apologies. You're right, Han-Teng. The reviewer looks to be Piotr Konieczny who I think is on this mailing list?

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 12:58, h hanteng@gmail.com wrote:

...

Heather, I am not sure who contribute that. Probably not Nemo. If this issue of newsletter is correctly attributed, the contributors include: Taha Yasseri, Maximilian Klein, Piotr Konieczny, Kim Osman, and Tilman Bayer. My suggestion is only a personal one, and I am not sure if it is against policies to make a few edits once the newsletter is out.

Thanks again to the contributors of the newsletter, my life is a bit easier and more interesting because of your work.

2014-07-02 15:35 GMT+07:00 Heather Ford hfordsa@gmail.com:

+1 Thanks for your really thoughtful comments, Joe, Han-Teng.

...

Nemo, would you be willing to add a note to the review and/or contacting the researcher?

Best, Heather.

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 05:17, h hanteng@gmail.com wrote:

...

The tone of the sentence in question

'it is disappointing that the main purpose appears to be

completing a thesis, with little thought to actually improving Wikipedia'

could have been written as

'It would be more useful for the Wikipedia community of practice

if the author discussed or even spelled out the implications of the research for improving Wikipedia".

This suggestion is based on my own impression that [Wiki-research-l]

has mainly two groups of readers: community of practice and community of knowledge. It is okay to have some group tensions for creative/critical inputs. Still, a neutral tone is better for assessment, and an encouraging tone might work a bit better to encourage others to fill the *gaps* (both practice and knowledge ones).

Also, the factors such as originally intended audience and word

limits may determine how much a writer can do for *due weight* (similar to [[WP:due]]). If the original (academic) author failed to address the implications for practices satisfactory, a research newsletter contributor can point out what s/he thinks the potential/actual implications are. (My thanks to the research newsletter's voluntary contributors for their unpaid work!)

While I understand that the monthly research newsletter has its

own perspective and interests different from academic newsletters, it does not sacrifice the integrity of the newsletter to be gentle and specific. I would recommend a minor edit to the sentence as the the newsletter could be read by any one in the world, not just the Wikipedians. It is public/published for all readers, and thus please do not assume the readers know the context of Wikipedia research.

Best,

han-teng liao

2014-07-01 19:37 GMT+07:00 Heather Ford hfordsa@gmail.com:

...

Thanks so much for the newsletter [1]! Always a great read...

But have to just say that comments like this: 'it is disappointing that the main purpose appears to be completing a thesis, with little thought to actually improving Wikipedia' [2] are really harsh and a little unfair. The student is studying Wikipedia - they can hardly only be interested in completing their thesis. We need to remember that researchers are at very different stages of their careers, they have very different motivations, and different levels of engagement with the Wikipedia community, but that *all* research on Wikipedia contributes to our understanding (even if as a catalyst for improvements). We want to encourage more research on Wikipedia, not attack the motivations of people we know little about - particularly when they're just students and particularly when this newsletter is on housed on Wikimedia Foundation's domain.

Best, Heather.

[1] https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June [2] https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June#.22Recommendin...

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk/ Doctoral Programme EthnographyMatters http://ethnographymatters.net/ | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115

http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

The newsletter is a Wiki-based project. Anyone with Internet access can edit it, so if someone doesn't find a word or sentence quite ok, should go and edit it to make it better, nicer, more accurate, fairer, etc... You even don't need to email it here, just go and change it. Your contributions are always very welcome.. (well, please do it before the release of the issue, but in few cases we have changed even after the release, Tilman knows the best about this). For the same reason, adding the names to individual reviews is not relevant. Many reviews that I have contributed to during the last 3 years, have not been written solely by me, again they have been written collaboratively, just like Wikipedia [1]. We don't add our names to our contributions to Wikipedia either (I suppose).

Cheers, Taha [1]: http://en.wikipedia.org/wiki/Wikipedia

On Thu, Jul 3, 2014 at 2:09 PM, Oliver Keyes okeyes@wikimedia.org wrote:

Both of these suggestions sound great to me! I'm not sure who the best person is to move them forward (I encourage anyone who wants to volunteer to speak up!) but whatever happens, I'm really grateful that we could turn this into a 'how do we fix this in the long-term?' conversation and not get bogged down - it's one of the most productive mailing list threads I've seen in a while :)

On Thursday, 3 July 2014, Heather Ford hfordsa@gmail.com wrote:

...

Thanks so much for this, Kerry. And thanks, Aaron for (as always) great, productive suggestions.

I think there are two issues that need to be dealt with separately here. The first is about disparaging remarks made about researchers' contributions that kicked off this discussion. One idea that I had when I saw a similar problem earlier this year was to at least have reviewers add their names to reviews so that we are making a clear distinction between the opinion of a single reviewer and the community/organisation as a whole. Some reviewers have added their names to reviews (thank you!) but I think that needs to be a standard for the newsletter. This probably won't solve the problem completely but hopefully reviewers will be more thoughtful about their critique in the future.

The second is to encourage research about Wikipedia that engages with the Wikimedia community. And yes, I, too, think that awards and acknowledgements are great ideas. I'd say that, when evaluating, engagement is even more important than impact because we want to encourage students and researchers at various stages of their careers (many of whom would not win awards for impact) to engage with the community when working on these projects. Of course, this kind of work is necessarily going to have more impact because Wikimedians themselves are going to be a part of it somehow. For this, I definitely agree with some kind of acknowledgement of research done - beyond, perhaps, just one or two star researchers winning a few awards. This can be done together e.g. awards for best papers in different categories but also acknowledgements for work with the community on particular projects as suggested by Kerry.

Best, Heather.

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 3 July 2014 02:56, Kerry Raymond kerry.raymond@gmail.com wrote:

...

Having had a work role oversighting many university researchers including PHD and other research students, I think many start out with intentions to engage fully with stakeholders and contribute back into the real world in some way, but it's fair to say that deadline pressures tend to force them to focus their energies into the "academically valued" outcomes, e.g. published papers, theses, etc. This is just as true for Wikipedia-related research as for, say, aquaculture. Of course, some never intended to contribute back, but are solely motivated by climbing the greasy pole of academia.

Because data gathering can be a time-consuming or expensive stumbling block in a research plan, organisations that freely publish detailed data (as WMF does) are natural magnets to researchers who can use that data to study various phenomena which may have broader relevance than just Wikipedia or where the Wikipedia data serves as a ground truth for other experiments or as proxy for other unavailable data. For example, you can use Wikipedia to study categorisation or named entity extraction without having real interest in Wikipedia itself.

So I think it is for those who are passionate about Wikipedia itself to see how such research findings may be used to improve Wikipedia. As for releasing source code, it has to recognised that software in research projects is often very quick-and-dirty and probably not designed to be integrated into the MediaWiki code base. Effective solutions to Wikipedia issues often require a mix of technology and change to community process/culture (which is often far harder to get right).

This is not to say they we should not encourage researchers to "give back", but I think we do need to understand that the reasons people don't give back aren't always attributable solely to "bad faith".

In additions to suggestions already made re awards, just having a letter of commendation on WMF letterhead acknowledging the research and its potential to improve Wikipedia would be a useful thing especially for junior researchers seeking to establish themselves; this kind of external validation is helpful to their CVs. This could be sent to any researchers whose research was deemed to have merit with different wording for those who made (according to some appropriately-appointed group) greater or lesser contributions to real Wikipedia impact.

Sent from my iPad

On 3 Jul 2014, at 12:15 am, Aaron Halfaker aaron.halfaker@gmail.com wrote:

Given that it seems we agree with Poitr's desire for research about Wikipedia to lead to useful tools an insights that can be directly applied to making Wikipedia and other wikis better, what might be a more effective strategy for encouraging researchers to engage with us or at least release their work in forms that we can more easily work with?

Here's a couple of half-baked ideas:

• *Wiki research impact task force* -- contacts authors to encourage

them to release code/datasets/etc. and praise them publicly when they do -- could be part of the work of newsletter reviewers. There are many researchers on this list who work directly with Wikimedians to make sure that their research has direct impact and their awesomeness is worth our appreciation and public recognition.

• *Yearly research award* -- for the most directly impactful

research projects/researchers similar to https://meta.wikimedia.org/wiki/Research:Wikimedia_France_Research_Award. One of the focuses of the judging could be the direct impact that the work has had.

-Aaron

On Wed, Jul 2, 2014 at 7:05 AM, Heather Ford hfordsa@gmail.com wrote:

...

Apologies. You're right, Han-Teng. The reviewer looks to be Piotr Konieczny who I think is on this mailing list?

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 12:58, h hanteng@gmail.com wrote:

...

Heather, I am not sure who contribute that. Probably not Nemo. If this issue of newsletter is correctly attributed, the contributors include: Taha Yasseri, Maximilian Klein, Piotr Konieczny, Kim Osman, and Tilman Bayer. My suggestion is only a personal one, and I am not sure if it is against policies to make a few edits once the newsletter is out.

Thanks again to the contributors of the newsletter, my life is a bit easier and more interesting because of your work.

2014-07-02 15:35 GMT+07:00 Heather Ford hfordsa@gmail.com:

+1 Thanks for your really thoughtful comments, Joe, Han-Teng.

...

Nemo, would you be willing to add a note to the review and/or contacting the researcher?

Best, Heather.

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115

http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 2 July 2014 05:17, h hanteng@gmail.com wrote:

> The tone of the sentence in question > > 'it is disappointing that the main purpose appears to be > completing a thesis, with little thought to actually improving > Wikipedia' > > could have been written as > > 'It would be more useful for the Wikipedia community of practice > if the author discussed or even spelled out the implications of the > research for improving Wikipedia". > > This suggestion is based on my own impression that [Wiki-research-l] > has mainly two groups of readers: community of practice and community of > knowledge. It is okay to have some group tensions for creative/critical > inputs. Still, a neutral tone is better for assessment, and an encouraging > tone might work a bit better to encourage others to fill the *gaps* (both > practice and knowledge ones). > > Also, the factors such as originally intended audience and word > limits may determine how much a writer can do for *due weight* (similar to > [[WP:due]]). If the original (academic) author failed to address the > implications for practices satisfactory, a research newsletter contributor > can point out what s/he thinks the potential/actual implications are. (My > thanks to the research newsletter's voluntary contributors for > their unpaid work!) > > While I understand that the monthly research newsletter has its > own perspective and interests different from academic newsletters, it does > not sacrifice the integrity of the newsletter to be gentle and specific. I > would recommend a minor edit to the sentence as the the newsletter could be > read by any one in the world, not just the Wikipedians. It is > public/published for all readers, and thus please do not assume the readers > know the context of Wikipedia research. > > Best, > > han-teng liao > > > 2014-07-01 19:37 GMT+07:00 Heather Ford hfordsa@gmail.com: > >> Thanks so much for the newsletter [1]! Always a great read... >> >> But have to just say that comments like this: 'it is disappointing >> that the main purpose appears to be completing a thesis, with >> little thought to actually improving Wikipedia' [2] are really harsh and a >> little unfair. The student is studying Wikipedia - they can hardly only be >> interested in completing their thesis. We need to remember that researchers >> are at very different stages of their careers, they have very different >> motivations, and different levels of engagement with the Wikipedia >> community, but that *all* research on Wikipedia contributes to our >> understanding (even if as a catalyst for improvements). We want to >> encourage more research on Wikipedia, not attack the motivations of people >> we know little about - particularly when they're just students and >> particularly when this newsletter is on housed on Wikimedia Foundation's >> domain. >> >> Best, >> Heather. >> >> [1] https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June >> [2] >> https://meta.wikimedia.org/wiki/Research:Newsletter/2014/June#.22Recommendin... >> >> Heather Ford >> Oxford Internet Institute http://www.oii.ox.ac.uk/ Doctoral >> Programme >> EthnographyMatters http://ethnographymatters.net/ | Oxford >> Digital Ethnography Group >> http://www.oii.ox.ac.uk/research/projects/?id=115 >> http://hblog.org | @hfordsa http://www.twitter.com/hfordsa >> >> >> _______________________________________________ >> Wiki-research-l mailing list >> Wiki-research-l@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >> >> > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > >

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

-- Sent from a portable device of Lovecraftian complexity.

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

As someone who is more of a free rider to the newsletter (*blushes*), I might not have as much of a say in this, but I agree that a peer review is different than an encyclopedia article. I'd like to think that a peer review or editorial doesn't proceed according to the same kind of more-or-less universal standards and processes that govern encyclopedic work, and thus make it modular enough to support a collective author function.

And there are many spaces in Wikipedia (and thanks for the cite, Taha :P) where individual editors write statements themselves, where it is incredibly useful to represent the opinion of a single individual. The Signpost [0] has bylines. Beyond that, Articles for Creation reviews [1] and Third Opinion [2] are probably the best analogies to research newsletter reviews, but this also takes place in X for deletion nominations, Arbitration Committee statements, and more.

I think in the grand scheme of things, it is actually productive to have somebody be able to say something a bit more opinionated like "it is disappointing that the main purpose appears to be completing a thesis, with little thought to actually improving Wikipedia", because as we see here, that leads to fruitful conversations. But I do think that they should be attributed to the opinion of an individual, who can then further discuss this opinion publicly.

There is certainly room for collaboration in reviews, and there can be co-authored reviews. However, if someone is listed as an author, then maybe they then have a kind of responsibility for the content. If they just edited it for grammar, etc., then perhaps they can be listed as an editor. Just some thoughts.

Also, I totally agree with the research awards, perhaps more thoughts on those later.

Best, Stuart

[0] http://en.wikipedia.org/wiki/Wikipedia:Signpost [1] http://en.wikipedia.org/wiki/Wikipedia:Articles_for_creation [2] http://en.wikipedia.org/wiki/Wikipedia:3O

On Thu, Jul 3, 2014 at 8:58 AM, Maximilian Klein isalix@gmail.com wrote:

Taha, even though the newsletter sections are a Wiki written by multiple people, we could still add multiple names in the "by-line." Do you see a problem with that?

We are not writing an Enclyclopedia here, but a research newsletter (it just happens to be hosted on an encyclopedia server). I think that for "intellectual honesty" it was a good idea of Heather's to add names to reviews. IMHO, we have an obligation to be even more rigorous in the newsletter than in writing Wikipedia.

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

You're right, Stuart. Having a byline (and not worrying so much about what is said) is probably enough because it would be clear who is speaking.

I have reviewed in the past and want to start again now that I have a bit more time. Dario, Tilman, you usually let us know when things need to be reviewed on this list, right? Perhaps we can do something similar when the newsletter is ready for a last proof as Joe suggests. And since I've been so opinionated, I will chat to others to try to help out streamline it a bit more because I know that everyone is really pressed for time when it comes to the newsletter. It's so great and important that I'm sure we can all help out a bit more :)

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 3 July 2014 17:58, Joe Corneli holtzermann17@gmail.com wrote:

On Thu, Jul 3, 2014 at 2:50 PM, Taha Yasseri taha.yaseri@gmail.com wrote:

...

Your contributions are always very welcome.. (well, please do it before the release of the issue, but in few cases we have changed even

after

...

the release, Tilman knows the best about this).

I've just subscribed to the newsletter as a mailing list - via https://lists.wikimedia.org/mailman/listinfo/research-newsletter

... But perhaps it would be useful to have a pre-release version of the mailing list, that would send it out a day or two in advance of the "official" release to persons who might be interested to help edit (or at least proofread)?

(I realize this might sound like crazy talk, but it's meant as a serious suggestion.)

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

I've been thinking about this and I want to make it clear what I'm proposing:

* that we make a rule/standard/style that people writing substantive reviews (i.e. reviews beyond short summaries where the opinion of the review is clearly reflected) be accompanied by a byline underneath the headline i.e.

'New study shows Wikipedia as powerful new gatekeeper Heather Ford

A new study by Anna Awesomepants has found that....'

The nature of the newsletter is such that the work is most often divided so that individual authors write reviews of individual articles, but if there are cases where more than one person has reviewed an article, then both names can be added. I think the reviews need to be attributed with real names, especially if people are critiquing the work of named individuals.

It has been suggested in the past that anyone who wants to add their name to their review should just do so but that it doesn't have to be required. This is problematic because there will still be unattributed reviews - and often those reviews are the problematic ones. Another suggestion has been that I oversee this process when the newsletter is developed. I don't mind doing this once or twice but I want this to be a rule/standard/style agreed to by this community so that Tilman, when he sets up the etherpad for the month can simply write at the top of the pad:

'Please write your name next to your review.'

I'm not always going to be able to review for the newsletter. Tilman and Dario coordinate this every month, but they need to be given a clear mandate. I'd rather make this explicit. I know that we're often afraid of rules in this community, but there are always rules - the difference is whether they're hidden or explicit. At least with the explicit ones we know how to oppose, comply with or add to them.

Then, a few responses to issues raised here:

Why looking at the edit history is not sufficient as attribution:

There are plenty of reasons why edit history does not serve as sufficient attribution.

a) Many reviews are actually produced in the etherpad before Tilman ports them over onto the wiki in which case the reviewer's name will not be visible.

b) More importantly, there are good reasons why Wikipedia uses this method for attributing authors of articles which are not relevant to the newsletter. Not every product works like Wikipedia; nor should it. Wikipedia attributes opinions to reliable sources whereas what we're doing here is 'original research'. In Wikipedia, the source is always supposed to be named. The words: 'it is disappointing that the researcher didn't release their code' wouldn't legitimately appear in a Wikipedia article. Instead, it would look something like this: 'According to Rev Researcher <cite>, 'It is disappointing that...' Or even better, 'according to some researchers <cite researchers A, B, C>...' but then the requirement is for more than one individual with a reputation in their community of expertise to be cited by name (not username or IP address but real name).

There are good reasons why we want to enable reviewers to assert their own opinion (preferably in a manner that is respectful and with the view to building relationships with researchers rather than alienating them). But then we need to have the academic integrity to attribute our opinions in order to invite dialogue with them.

Best, Heather.

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 3 July 2014 21:17, Taha Yasseri taha.yaseri@gmail.com wrote:

Thanks Stuart, Max, and Heather, But let's keep things simple and efficient (as it is right now). If we want to use bylines for all the contributions, then the next question would be whether we have to use the real names or Wikipedia user names or even IP addresses would be enough or not (IP address is enough in some of Stuart's examples).

Of course if someone wants to add their name to the review, it should be allowed (as it is now), but it also doesn't mean that others can not edit that review.

Also to address concerns about the sentiment and fairness of the reviews (which is a valid concern in general), again, everyone is welcome to have a look at the draft and the pre-release version to make sure that all the reviews are at a conventional quality. Usually Dario and Tilman send a link to the draft few days before the release and that's the best time for action.

Best, Taha

On Thu, Jul 3, 2014 at 8:50 PM, Heather Ford hfordsa@gmail.com wrote:

...

You're right, Stuart. Having a byline (and not worrying so much about what is said) is probably enough because it would be clear who is speaking.

I have reviewed in the past and want to start again now that I have a bit more time. Dario, Tilman, you usually let us know when things need to be reviewed on this list, right? Perhaps we can do something similar when the newsletter is ready for a last proof as Joe suggests. And since I've been so opinionated, I will chat to others to try to help out streamline it a bit more because I know that everyone is really pressed for time when it comes to the newsletter. It's so great and important that I'm sure we can all help out a bit more :)

Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa

On 3 July 2014 17:58, Joe Corneli holtzermann17@gmail.com wrote:

...

On Thu, Jul 3, 2014 at 2:50 PM, Taha Yasseri taha.yaseri@gmail.com wrote:

...

Your contributions are always very welcome.. (well, please do it before the release of the issue, but in few cases we have changed even

after

...

the release, Tilman knows the best about this).

I've just subscribed to the newsletter as a mailing list - via https://lists.wikimedia.org/mailman/listinfo/research-newsletter

... But perhaps it would be useful to have a pre-release version of the mailing list, that would send it out a day or two in advance of the "official" release to persons who might be interested to help edit (or at least proofread)?

(I realize this might sound like crazy talk, but it's meant as a serious suggestion.)

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

-- .t

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

There are plenty of reasons why edit history does not serve as sufficient attribution.

As it seems to be constantly forgotten in this thread: the name of the author was *not* left to the history, it was linked in the sentence "This reviewer thinks". I use to hover links so for me it was as visible as a byline.

Nemo

Stuart -- You make good points ('render unto academia what is academia's). But I still think further personalization and even clearer attribution would have gone a long way...

'it is disappointing that the main purpose appears to be completing a thesis, with little thought to actually improving Wikipedia'

====>

'This reviewer [again linked] is disappointed that the main purpose appears to be completing a thesis, with little thought to actually improving Wikipedia' or even more strongly 'I (username)...'

To me that's a vivid transformation.

----- Further--a thought about impact: If we want to make research more impactful, another place to look is the ease with which researchers can test (bad as well as good) ideas. The Wikipedia community has been over-researched: do we *really* want to encourage every MSc student on a 1-year thesis project to engage? Can we, as a research community, facilitate that, if so?

-Jodi

PS-As far as I can tell, this project is a really keen idea -- and perhaps the door is now open for *somebody* to translate the research? http://hci.cs.umanitoba.ca/projects-and-research/details/intelwiki

On Mon, Jul 7, 2014 at 12:34 AM, Stuart A. Yeates syeates@gmail.com wrote:

I've been avoiding jumping into this thread, to let people closer to the issue have the first say but it seems to me that there are a couple of things that bear saying:

• We're a cross-discipline group, academia and Wikipedia

• While the portion of the review in question may not have been an

appropriate academic criticism, it was certainly an appropriate Wikipedia criticism (and a criticism I agree with).

• It's up to those who write it to collectively to decide what the

newsletter to be. Deference to the standards of academia will benefit the careers of those in academia. Deference to the standards of Wikipedia will increase the chances of some of this research actually leading to better outcomes in live wiki. Maybe a better articulation of this to reviewers and reviewed might help, as might two-part reviews addressing the concerns of each audience separately.

• I can't believe that there's a shortage of people to write reviews.

I can believe that there's a shortage of people motivated to write reviews. Maybe we could look at a DYK-like quid pro quo system? Note that this could be done independently from the editing of the newsletter, all it would take is a quorum of (potential) editors to set up a wiki page to coordinate and set standards.

cheers stuart

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l