I'd be happy to offer my admin/oversighter
experience and knowledge to help
you develop the labeling and such, Aaron! I just commented on Andreas's
proposal on the Community Wishlist, but to summarize here: I see a lot of
potential pitfalls in trying to handle/generalize this with machine
learning, but I also see a lot of potential value, and I think it's
something we should be investigating.
-Fluffernutter
On Sun, Nov 15, 2015 at 11:32 AM, Aaron Halfaker <ahalfaker(a)wikimedia.org>
wrote:
The League of Legends team collaborated with outside scientists to
analyse their dataset. I would love to see the Wikimedia Foundation
engage
in a similar research project.
Oh! We are! :) When we have time. :\ One of the projects that I'd like to
see done, but I've struggled to find the time for is a common talk page
parser[1] that could produce a dataset of talk page interactions. I'd like
this dataset to be easy to join to editor outcome measures. E.g. there
might be "aggressive" talk that we don't know is problematic until we see
the kind of effect that it has on other conversation participants.
Anyway, I want some powerful utilities and datasets out there to help
academics look into this problem more easily. For revscoring, I'd like to
be able to take a set of talk page diffs, have them classified in Wiki
labels[2] as "aggressive" and the build a model for ORES[3] to be used
however people see fit. You could then use ORES to do offline analysis of
discussions for research. You could use ORES to interrupt the a user
before saving a change. I'm sure there are other clever ideas that people
have for what to do with such a model that I'm happy to enable it via the
service. The hard part is getting a good dataset labeled.
If someone wants to invest some time and energy into this, I'm happy to
work with you. We'll need more than programming help. We'll need a lot of
help to figure out what dimensions we'll label talk page postings by and to
do the actual labeling.
1.
https://github.com/Ironholds/talk-parser
2.
https://meta.wikimedia.org/wiki/Wiki_labels
3.
https://meta.wikimedia.org/wiki/ORES
On Sun, Nov 15, 2015 at 6:56 AM, Andreas Kolbe <jayen466(a)gmail.com> wrote:
On Sat, Nov 14, 2015 at 9:13 PM, Benjamin Lees
<emufarmers(a)gmail.com>
wrote:
This article highlights the happier side of
things, but it appears
that Lin's approach also involved completely removing bad actors:
"Some players have also asked why we've taken such an aggressive
stance when we've been focused on reform; well, the key here is that
for most players, reform approaches are quite effective. But, for a
number of players, reform attempts have been very unsuccessful which
forces us to remove some of these players from League entirely."[0]
Thanks for the added context, Benjamin. Of course, banning bad actors
that
they consider unreformable is something Wikipedia
admins have always done
as well.
The League of Legends team began by building a dataset of interactions
that
the community considered unacceptable, and then
applied machine-learning
to
that dataset.
It occurs to me that the English Wikipedia has ready access to such a
dataset: it's the totality of revision-deleted and oversighted talk page
posts. The League of Legends team collaborated with outside scientists to
analyse their dataset. I would love to see the Wikimedia Foundation
engage
in a similar research project.
>
> I've added this point to the community wishlist survey:
>
>
>
https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey#Machine-lear…
P.S. As Rupert noted, over 90% of LoL players are
male (how much over
90%?).[1] It would be interesting to know whether this percentage has
changed along with the improvements described in the article.
Indeed.
_______________________________________________
Wikimedia-l mailing list, guidelines at:
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l(a)lists.wikimedia.org
Unsubscribe:
https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
<mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
_______________________________________________
Wikimedia-l mailing list, guidelines at:
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l(a)lists.wikimedia.org
Unsubscribe:
https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
<mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
--
Karen Brown
user:Fluffernutter
*Unless otherwise specified, any email sent from this address is in my
volunteer capacity and does not represent the views or wishes of the
Wikimedia Foundation*
_______________________________________________
Wikimedia-l mailing list, guidelines at: