I would love to see a grant from the WEF to pay humans to do Revision Scoring as a Service as per http://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service
Can someone please put that in IdeaLab and send me the URL?
On Thu, Apr 9, 2015 at 6:59 AM, Jason Radford jsradford@uchicago.edu wrote:
Hi Everyone,
I wanted to share two projects currently under consideration for IdeaLab funding and which may be of directly related to engaging new and seasoned editors. If you are interested or know someone who might be, let me know. If you have feedback for these projects, please submit it on their discussion pages.
Thanks, Jason
Wiki Controversy Monitoring Engine Call for Developers
Purpose: The controversy monitoring engine maintains a real-time rating of the controversiality of Wikipedia articles by listening to the live stream of edits from Wikipedia. We need someone who is interested in building the web interface and interactive visualizations around these controversies to enable administrators to monitor, investigate, and, if need be, intervene to deescalate controversies. The goal is to create a site like stats.wikimedia.org editors and admins can use to identify and deescalate controversies.
Requirements: Knowledge of web development, web-based visualization, and\or data analysis using Wikipedia's API or WikiData.
For More Information: see https://meta.wikimedia.org/wiki/Grants:IdeaLab/Controversy_Monitoring_Engine
Consciousness-Raising Repository Call for Working Group Participants
Purpose: We're putting together a group of diverse Wikipedians to help put together a repository of stories from users experiencing marginalization on Wikipedia. Whether first timers or old timers, we're looking to recruit users interested in engaging the community around marginalization. The purpose of the repository is to serve as a database of knowledge about the forms marginalization can take and as an outlet for users experiencing marginalization.
Requirements: Interest in working with marginalization and marginalized groups. Interest in the Wiki-community. Willing to attend an hour-long biweekly meeting.
For more information: https://meta.wikimedia.org/wiki/Grants:IdeaLab/A_Consciousness_Raising_Repos...
--
Jason Radford Doctoral Student, Sociology, University of Chicago Visiting Researcher, Lazer Lab, Northeastern University Connect: LinkedIn, Twitter, University of Chicago Play Games for Science at Volunteer Science
EE mailing list EE@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/ee
There is also another grant of interest from the Inspire campain to the Education program in general:
https://meta.wikimedia.org/wiki/Grants:IdeaLab/University_of_Nebraska_Women_...
On my volunteer time, I have been talking to Daniel (the grant nominator) about how to execute a program like this: and he seems to have created quite a coalition of interest at UNL. What's important about the coalition: he is going to be involving a number of people active in the Digital Humanities: a field that has overlapping interests and skills that would be useful for the Wikipedia Education program, but in which we are only starting to see Wikipedia approached as a tool of interest. I would recommend supporting the grant: because it provides us a model of involvement for Universities that isn't just bound by individual classrooms.
Alex Stinson
Alex Stinson
On Thu, Apr 9, 2015 at 7:30 PM, James Salsman jsalsman@gmail.com wrote:
I would love to see a grant from the WEF to pay humans to do Revision Scoring as a Service as per http://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service
Can someone please put that in IdeaLab and send me the URL?
On Thu, Apr 9, 2015 at 6:59 AM, Jason Radford jsradford@uchicago.edu wrote:
Hi Everyone,
I wanted to share two projects currently under consideration for IdeaLab funding and which may be of directly related to engaging new and seasoned editors. If you are interested or know someone who might be, let me
know.
If you have feedback for these projects, please submit it on their discussion pages.
Thanks, Jason
Wiki Controversy Monitoring Engine Call for Developers
Purpose: The controversy monitoring engine maintains a real-time rating
of
the controversiality of Wikipedia articles by listening to the live
stream
of edits from Wikipedia. We need someone who is interested in building
the
web interface and interactive visualizations around these controversies
to
enable administrators to monitor, investigate, and, if need be,
intervene to
deescalate controversies. The goal is to create a site like stats.wikimedia.org editors and admins can use to identify and
deescalate
controversies.
Requirements: Knowledge of web development, web-based visualization,
and\or
data analysis using Wikipedia's API or WikiData.
For More Information: see
https://meta.wikimedia.org/wiki/Grants:IdeaLab/Controversy_Monitoring_Engine
Consciousness-Raising Repository Call for Working Group Participants
Purpose: We're putting together a group of diverse Wikipedians to help
put
together a repository of stories from users experiencing marginalization
on
Wikipedia. Whether first timers or old timers, we're looking to recruit users interested in engaging the community around marginalization. The purpose of the repository is to serve as a database of knowledge about
the
forms marginalization can take and as an outlet for users experiencing marginalization.
Requirements: Interest in working with marginalization and marginalized groups. Interest in the Wiki-community. Willing to attend an hour-long biweekly meeting.
For more information:
https://meta.wikimedia.org/wiki/Grants:IdeaLab/A_Consciousness_Raising_Repos...
--
Jason Radford Doctoral Student, Sociology, University of Chicago Visiting Researcher, Lazer Lab, Northeastern University Connect: LinkedIn, Twitter, University of Chicago Play Games for Science at Volunteer Science
EE mailing list EE@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/ee
Education mailing list Education@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/education
On 10 April 2015 at 07:53, Alex Stinson sadads@gmail.com wrote:
What's important about the coalition: he is going to be involving a number
of people active in the Digital Humanities: a field that has overlapping interests and skills that would be useful for the Wikipedia Education program, but in which we are only starting to see Wikipedia approached as a tool of interest.
I would agree with Alex here. Digital Humanities (DH) means different things to different people, but combines "data led" with "reaches the parts others don't" for Wikimedia in an enticing way. It's a big discussion. Just one point: it helps to think of sister projects (Commons, Wikisource, Wikidata) coming together in the offering.
Charles
Hello everyone,
First, thank you, James for sharing and reviving interest in revision scoring. It is really an important aspect of education programs but unfortunately not many of the program volunteers around the world paid much attention to it before. I would also like to thank Alex Stinson for sharing that inspire campaign grant proposal with us. He was right in his endorsement mentioning the Wiki Education Foundation as they take care of/manage Wikipedia Education Program in the US and Canada. They would be the best people to contact concerning Wikipedia education activities there.
It was great that you thought of IdeaLab https://meta.wikimedia.org/wiki/Grants:IdeaLab, James! We always say "*Be bold* https://en.wikipedia.org/wiki/Wikipedia:Be_bold" which means that it would be better to see your ideas expressed on wiki by your hands, so, please be bold, start the IdeaLab page and share the link with us.
Please don't hesitate to contact me with any questions you may have.
Sincerely,
Samir
On Fri, Apr 10, 2015 at 9:21 AM, Charles Matthews < charles.r.matthews@ntlworld.com> wrote:
On 10 April 2015 at 07:53, Alex Stinson sadads@gmail.com wrote:
What's important about the coalition: he is going to be involving a number
of people active in the Digital Humanities: a field that has overlapping interests and skills that would be useful for the Wikipedia Education program, but in which we are only starting to see Wikipedia approached as a tool of interest.
I would agree with Alex here. Digital Humanities (DH) means different things to different people, but combines "data led" with "reaches the parts others don't" for Wikimedia in an enticing way. It's a big discussion. Just one point: it helps to think of sister projects (Commons, Wikisource, Wikidata) coming together in the offering.
Charles
Education mailing list Education@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/education
Thank you, Samir!
I need to see updates at
https://meta.wikimedia.org/wiki/Grants:IEG/Revision_scoring_as_a_service/Tim...
before I can know what to write. Does the WEF use the WMF's Meta IdeaLab for grants, too?
Also, I need a login to be able to read http://wikiedu.org/wiki/index.php?title=Outreach_Pilot
Would you please send me a login to that wiki? I will happily set up a page with a proposal to pay people to do revision scoring. I would also like to propose paying people to do
https://www.mediawiki.org/w/index.php?title=Accuracy_review
which is like revision scoring, but targeted to things that are most likely to be wrong, make the most difference when they're wrong, confuse people when they're wrong, etc. Revision scoring also does this, but in a potentially less targeted fashion. There may also be a happier medium between the two approaches, but it seems unlikely that people will be able to figure out what the space of mediums is without constructing the endpoints.
Best regards, James
On Mon, Apr 13, 2015 at 5:22 AM, Samir Elsharbaty selsharbaty@wikimedia.org wrote:
Hello everyone,
First, thank you, James for sharing and reviving interest in revision scoring. It is really an important aspect of education programs but unfortunately not many of the program volunteers around the world paid much attention to it before. I would also like to thank Alex Stinson for sharing that inspire campaign grant proposal with us. He was right in his endorsement mentioning the Wiki Education Foundation as they take care of/manage Wikipedia Education Program in the US and Canada. They would be the best people to contact concerning Wikipedia education activities there.
It was great that you thought of IdeaLab, James! We always say "Be bold" which means that it would be better to see your ideas expressed on wiki by your hands, so, please be bold, start the IdeaLab page and share the link with us.
Please don't hesitate to contact me with any questions you may have.
Sincerely,
Samir
On Fri, Apr 10, 2015 at 9:21 AM, Charles Matthews charles.r.matthews@ntlworld.com wrote:
On 10 April 2015 at 07:53, Alex Stinson sadads@gmail.com wrote:
What's important about the coalition: he is going to be involving a number of people active in the Digital Humanities: a field that has overlapping interests and skills that would be useful for the Wikipedia Education program, but in which we are only starting to see Wikipedia approached as a tool of interest.
I would agree with Alex here. Digital Humanities (DH) means different things to different people, but combines "data led" with "reaches the parts others don't" for Wikimedia in an enticing way. It's a big discussion. Just one point: it helps to think of sister projects (Commons, Wikisource, Wikidata) coming together in the offering.
Charles
Education mailing list Education@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/education
-- Samir Elsharbaty, Communications Intern, Wikipedia Education Program Wikimedia Foundation +2.011.200.696.77 selsharbaty@wikimedia.org education.wikimedia.org
Education mailing list Education@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/education
Wiki Ed has no plans to apply for a grant for the Inspire campaign -- as an independent 501(c)3 nonprofit, we have other funding opportunities to support our programs that tackle the gender gap, and we don't want to compete for movement funds in this area with other groups who have a less well-developed fundraising capability than we do.
But we obviously support the work to close the gender gap -- we've steadily maintained higher than 60% female student editors (68% last term! http://wikiedu.org/blog/2015/02/05/gender-gap-68-percent/), and our partnership with the National Women's Studies Association ( http://www.nwsa.org/content.asp?pl=17&contentid=111) is specifically targeting filling content gaps related to the gender gap.
And a housekeeping note -- the wikiedu.org wiki you linked to is our office wiki, which contains some sensitive information, so access is limited to Wiki Ed staff. But you can track more information about our programs in our monthly reports: https://meta.wikimedia.org/wiki/Wiki_Education_Foundation#Monthly_reports
Specifically related to the Outreach Pilot you mentioned, more information on the pilot (beyond what's said in each monthly report) is available in these blog posts: http://wikiedu.org/blog/category/outreach-program/
LiAnna
On Fri, Apr 17, 2015 at 11:05 AM, James Salsman jsalsman@gmail.com wrote:
Thank you, Samir!
I need to see updates at
https://meta.wikimedia.org/wiki/Grants:IEG/Revision_scoring_as_a_service/Tim...
before I can know what to write. Does the WEF use the WMF's Meta IdeaLab for grants, too?
Also, I need a login to be able to read http://wikiedu.org/wiki/index.php?title=Outreach_Pilot
Would you please send me a login to that wiki? I will happily set up a page with a proposal to pay people to do revision scoring. I would also like to propose paying people to do
https://www.mediawiki.org/w/index.php?title=Accuracy_review
which is like revision scoring, but targeted to things that are most likely to be wrong, make the most difference when they're wrong, confuse people when they're wrong, etc. Revision scoring also does this, but in a potentially less targeted fashion. There may also be a happier medium between the two approaches, but it seems unlikely that people will be able to figure out what the space of mediums is without constructing the endpoints.
Best regards, James
On Mon, Apr 13, 2015 at 5:22 AM, Samir Elsharbaty selsharbaty@wikimedia.org wrote:
Hello everyone,
First, thank you, James for sharing and reviving interest in revision scoring. It is really an important aspect of education programs but unfortunately not many of the program volunteers around the world paid
much
attention to it before. I would also like to thank Alex Stinson for
sharing
that inspire campaign grant proposal with us. He was right in his endorsement mentioning the Wiki Education Foundation as they take care of/manage Wikipedia Education Program in the US and Canada. They would be the best people to contact concerning Wikipedia education activities
there.
It was great that you thought of IdeaLab, James! We always say "Be bold" which means that it would be better to see your ideas expressed on wiki
by
your hands, so, please be bold, start the IdeaLab page and share the link with us.
Please don't hesitate to contact me with any questions you may have.
Sincerely,
Samir
On Fri, Apr 10, 2015 at 9:21 AM, Charles Matthews charles.r.matthews@ntlworld.com wrote:
On 10 April 2015 at 07:53, Alex Stinson sadads@gmail.com wrote:
What's important about the coalition: he is going to be involving a number of people active in the Digital Humanities: a field that has overlapping interests and skills that would be useful for the Wikipedia Education program, but in which we are only starting to see Wikipedia approached as a tool of interest.
I would agree with Alex here. Digital Humanities (DH) means different things to different people, but combines "data led" with "reaches the
parts
others don't" for Wikimedia in an enticing way. It's a big discussion.
Just
one point: it helps to think of sister projects (Commons, Wikisource, Wikidata) coming together in the offering.
Charles
Education mailing list Education@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/education
-- Samir Elsharbaty, Communications Intern, Wikipedia Education Program Wikimedia Foundation +2.011.200.696.77 selsharbaty@wikimedia.org education.wikimedia.org
Education mailing list Education@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/education
Education mailing list Education@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/education
Great; thank you, LeAnna.
I wish I could think of a budget for Accuracy Review. It would really help if someone from WEF would co-mentor it. LeAnna, would you like to co-mentor, or do you know anyone at Wiki Ed who would? I have to reach out to the Simple English Wikipedia and inform them I am asking WEF to help make a bot for them. Luckily, my experience with up-goer five talk, LOGLAN, Freudenthal's 1960 LINCOS, and English should make that easy. If you want to co-mentor, you can try that, or tell me to do it as you prefer.
Would it be okay to ask you to reach out to the Revision Scoring as a Service people and ask them, if you paid people to score revisions, how much you should offer? Although, it's a perfectly legitimate question to ask if that would put their (WMF's) fair harbor provisions at risk. I doubt it would, so I'll ignore the possibility for now. Please correct me if I'm wrong. If I had to guess at the starting amount, it would be $20 per hour plus pension and benefits. I don't know if that's right so I would love to hear other opinions.
I note that you can select a 50% ratio, and potentially more women than men, if you can amortize reparations. I have also been trying to work on attracting female editors within the strict confines of improving the encyclopedia. I predict we will have gender equality among editors by total attention by 2025 without accuracy review and by 2023 with it, but please don't put that in the advantages column until I've double checked it.
Best regards, James
On Friday, April 17, 2015, LiAnna Davis lianna@wikiedu.org wrote:
Wiki Ed has no plans to apply for a grant for the Inspire campaign -- as an independent 501(c)3 nonprofit, we have other funding opportunities to support our programs that tackle the gender gap, and we don't want to compete for movement funds in this area with other groups who have a less well-developed fundraising capability than we do.
But we obviously support the work to close the gender gap -- we've steadily maintained higher than 60% female student editors (68% last term! http://wikiedu.org/blog/2015/02/05/gender-gap-68-percent/), and our partnership with the National Women's Studies Association ( http://www.nwsa.org/content.asp?pl=17&contentid=111) is specifically targeting filling content gaps related to the gender gap.
And a housekeeping note -- the wikiedu.org wiki you linked to is our office wiki, which contains some sensitive information, so access is limited to Wiki Ed staff. But you can track more information about our programs in our monthly reports: https://meta.wikimedia.org/wiki/Wiki_Education_Foundation#Monthly_reports
Specifically related to the Outreach Pilot you mentioned, more information on the pilot (beyond what's said in each monthly report) is available in these blog posts: http://wikiedu.org/blog/category/outreach-program/
LiAnna
On Fri, Apr 17, 2015 at 11:05 AM, James Salsman <jsalsman@gmail.com javascript:_e(%7B%7D,'cvml','jsalsman@gmail.com');> wrote:
Thank you, Samir!
I need to see updates at
https://meta.wikimedia.org/wiki/Grants:IEG/Revision_scoring_as_a_service/Tim...
before I can know what to write. Does the WEF use the WMF's Meta IdeaLab for grants, too?
Also, I need a login to be able to read http://wikiedu.org/wiki/index.php?title=Outreach_Pilot
Would you please send me a login to that wiki? I will happily set up a page with a proposal to pay people to do revision scoring. I would also like to propose paying people to do
https://www.mediawiki.org/w/index.php?title=Accuracy_review
which is like revision scoring, but targeted to things that are most likely to be wrong, make the most difference when they're wrong, confuse people when they're wrong, etc. Revision scoring also does this, but in a potentially less targeted fashion. There may also be a happier medium between the two approaches, but it seems unlikely that people will be able to figure out what the space of mediums is without constructing the endpoints.
Best regards, James
On Mon, Apr 13, 2015 at 5:22 AM, Samir Elsharbaty <selsharbaty@wikimedia.org javascript:_e(%7B%7D,'cvml','selsharbaty@wikimedia.org');> wrote:
Hello everyone,
First, thank you, James for sharing and reviving interest in revision scoring. It is really an important aspect of education programs but unfortunately not many of the program volunteers around the world paid
much
attention to it before. I would also like to thank Alex Stinson for
sharing
that inspire campaign grant proposal with us. He was right in his endorsement mentioning the Wiki Education Foundation as they take care of/manage Wikipedia Education Program in the US and Canada. They would
be
the best people to contact concerning Wikipedia education activities
there.
It was great that you thought of IdeaLab, James! We always say "Be bold" which means that it would be better to see your ideas expressed on wiki
by
your hands, so, please be bold, start the IdeaLab page and share the
link
with us.
Please don't hesitate to contact me with any questions you may have.
Sincerely,
Samir
On Fri, Apr 10, 2015 at 9:21 AM, Charles Matthews <charles.r.matthews@ntlworld.com
javascript:_e(%7B%7D,'cvml','charles.r.matthews@ntlworld.com');> wrote:
On 10 April 2015 at 07:53, Alex Stinson <sadads@gmail.com
javascript:_e(%7B%7D,'cvml','sadads@gmail.com');> wrote:
What's important about the coalition: he is going to be involving a number of people active in the Digital Humanities: a field that has overlapping interests and skills that would be useful for the
Wikipedia
Education program, but in which we are only starting to see Wikipedia approached as a tool of interest.
I would agree with Alex here. Digital Humanities (DH) means different things to different people, but combines "data led" with "reaches the
parts
others don't" for Wikimedia in an enticing way. It's a big discussion.
Just
one point: it helps to think of sister projects (Commons, Wikisource, Wikidata) coming together in the offering.
Charles
Education mailing list Education@lists.wikimedia.org
javascript:_e(%7B%7D,'cvml','Education@lists.wikimedia.org');
-- Samir Elsharbaty, Communications Intern, Wikipedia Education Program Wikimedia Foundation +2.011.200.696.77 selsharbaty@wikimedia.org
javascript:_e(%7B%7D,'cvml','selsharbaty@wikimedia.org');
education.wikimedia.org
Education mailing list Education@lists.wikimedia.org
javascript:_e(%7B%7D,'cvml','Education@lists.wikimedia.org');
Education mailing list Education@lists.wikimedia.org javascript:_e(%7B%7D,'cvml','Education@lists.wikimedia.org'); https://lists.wikimedia.org/mailman/listinfo/education
-- LiAnna Davis Director of Programs Wiki Education Foundation +1-415-770-1061 www.wikiedu.org
I'll take the baton from LiAnna here, since this is a problem space I've been working on.
On Fri, Apr 17, 2015 at 3:16 PM, James Salsman jsalsman@gmail.com wrote:
I wish I could think of a budget for Accuracy Review. It would really help if someone from WEF would co-mentor it. LeAnna, would you like to co-mentor, or do you know anyone at Wiki Ed who would? I have to reach out to the Simple English Wikipedia and inform them I am asking WEF to help make a bot for them. Luckily, my experience with up-goer five talk, LOGLAN, Freudenthal's 1960 LINCOS, and English should make that easy. If you want to co-mentor, you can try that, or tell me to do it as you prefer.
We're broadly interested in the concept of 'accuracy review' (which would be quite different from the revision scoring, at least as laid out by Aaron Halfaker and company), but it's not something we've got the bandwidth for right now. We're also, at least for the time being, just focused on English Wikipedia.
Would it be okay to ask you to reach out to the Revision Scoring as a Service people and ask them, if you paid people to score revisions, how much you should offer? Although, it's a perfectly legitimate question to ask if that would put their (WMF's) fair harbor provisions at risk. I doubt it would, so I'll ignore the possibility for now. Please correct me if I'm wrong. If I had to guess at the starting amount, it would be $20 per hour plus pension and benefits. I don't know if that's right so I would love to hear other opinions.
I've been chatting with the folks working on this, and they are actually quite close to having a usable API for estimated article quality — which I'm super excited about building into our dashboard. The human part of it will be down the road a bit, but the main purpose there will be to continually improve the model by having experienced editors create good ratings data for training the model. But I expect that there won't be much trouble in finding Wikipedians to pitch on that.
I had actually been exploring the idea of setting up a crowdsourcing system where we might pay experienced editors to do before and after ratings for student work, but at this point I'm much more enthusiastic about the machine learning approach that the revision-scoring-as-a-service project is taking — since that is easy to scale up and maintain long term.
-Sage
Thank you, Sage, for your reply:
... I've been chatting with the folks working on this, and they are actually quite close to having a usable API for estimated article quality — which I'm super excited about building into our dashboard. The human part of it will be down the road a bit, but the main purpose there will be to continually improve the model by having experienced editors create good ratings data for training the model. But I expect that there won't be much trouble in finding Wikipedians to pitch on that.
I had actually been exploring the idea of setting up a crowdsourcing system where we might pay experienced editors to do before and after ratings for student work, but at this point I'm much more enthusiastic about the machine learning approach that the revision-scoring-as-a-service project is taking — since that is easy to scale up and maintain long term.
I recommend measuring the optimal amount of human input and review. It is very substantially nonzero if you want to maximize the encyclopedia's utility function. There is really nobody at the WEF who wants to try to co-mentor accuracy review? What if there was a cap on total hours needed. I am sure you wouldn't regret it, but I am also happy to continue on my own for the time being.
Best regards, James
On Fri, Apr 17, 2015 at 5:27 PM, James Salsman jsalsman@gmail.com wrote:
Thank you, Sage, for your reply:
... I've been chatting with the folks working on this, and they are
actually
quite close to having a usable API for estimated article quality — which
I'm
super excited about building into our dashboard. The human part of it
will
be down the road a bit, but the main purpose there will be to continually improve the model by having experienced editors create good ratings data
for
training the model. But I expect that there won't be much trouble in
finding
Wikipedians to pitch on that.
I had actually been exploring the idea of setting up a crowdsourcing
system
where we might pay experienced editors to do before and after ratings for student work, but at this point I'm much more enthusiastic about the
machine
learning approach that the revision-scoring-as-a-service project is
taking —
since that is easy to scale up and maintain long term.
I recommend measuring the optimal amount of human input and review. It is very substantially nonzero if you want to maximize the encyclopedia's utility function. There is really nobody at the WEF who wants to try to co-mentor accuracy review? What if there was a cap on total hours needed. I am sure you wouldn't regret it, but I am also happy to continue on my own for the time being.
I'm definitely interested in better systems for human review — especially for the work of student editors — alongside automated qualtiy estimation tools. It's not a project Wiki Ed has the capacity to take on right now, though.
-Sage