Hey all,
TLDR: ORES extension [1] which is an extension that integrates ORES service
[2] with Wikipedia to make fighting vandalism easier and more efficient is
in the progress of deployment. You can test it in
https://mw-revscoring.wmflabs.org (Enable it in your preferences first)
You probably know ORES. It's an API service that gives probably of an edit
being vandalism, it also does other AI-related stuff like guessing the
quality of articles in Wikipedia. We have a nice blog post in Wikimedia
Blog [3] and media paid some attention to it [4]. Thanks to Aaron Halfaker
and others [5] for their work in building this service. There are several
tools using ORES to highlight possibly vandalism edits. Huggle, gadgets
like ScoredRevisions, etc. But an extension does this job much more
efficiently.
The extension which is being developed by Adam Wight, Kunal Mehta and me
highlights unpatrolled edits in recentchanges, watchlists, related changes
and in future, user contributions if ORES score of those edits pass a
certain threshold. GUI design is made by May Galloway. ORES API (
ores.wmflabs.org) only gives you a score between 0 and 1. Zero means it's
not vandalism at all and one means it's vandalism for sure. You can test
its simple GUI in https://ores.wmflabs.org/ui/. It's possible to change the
threshold in your preferences in the recent changes tab (you have options
instead of numbers because we thought numbers are not very intuitive).
Also, we enabled it in a test wiki so you test it:
https://mw-revscoring.wmflabs.org. You need to make an account (use a dummy
password) and then enable it in beta features tab. Note that building AI
tool to detect vandalism in a test wiki sounds a little bit silly ;) so we
set up a dummy model that probability of an edit being vandalism is
backward of the last two digits (e.g. diff id:12345 = score:54%). In a more
technical aspect, we store these scores in ores_classification table so we
can do a lot more analysis with them once the extension is deployed. Fun
use cases such as the average score of a certain page or contributions of a
user or members of a category, etc.
We passed security review and we have consensus to enable it in Persian
Wikipedia. We are only blocked on ORES moving from Labs to production
(T106867 [6]). The next wiki is Wikidata, we are good to go once the
community finishes labeling edits so we can build the "damaging" model. We
can enable it Portuguese and Turkish Wikipedia after March because s2 and
s3 have database storage issues right now. For other Wikis, you need to
check if ORES supports the Wiki and if community finished labeling edits
for ORES (check out the table at [2])
If you want to report bugs or add feature requests you can find it in here
[7].
[1]: https://www.mediawiki.org/wiki/Extension:ORES
[2]: https://meta.wikimedia.org/wiki/Objective_Revision_Evaluation_Service
[3]:
https://blog.wikimedia.org/2015/11/30/artificial-intelligence-x-ray-specs/
[4]:
https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service/Media
[5]:
https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service#Team
[6]: https://phabricator.wikimedia.org/T106867
[7]: https://phabricator.wikimedia.org/tag/mediawiki-extensions-ores/
Best
Hey folks,
I just wanted to share a quick writeup that Andrew Lih did about using ORES
<https://meta.wikimedia.org/wiki/Objective_Revision_Evaluation_Service>'
article quality prediction model in the classroom to help students see the
impact of their work in Wikipedia. See
https://en.wikipedia.org/wiki/User:Fuzheado/ORES_experiment
Some quotes from the writeup:
“Yay!"
>
> “Alright!"
>
> "I feel like we’re part of a secret computer club!"
>
> [...] having students excited and experiencing that instant dopamine hit
> when editing articles is a breath of fresh air when pleasurable
> interactions seem to be less frequent today than in Wikipedia’s early days.
>
One important note is that I had *no idea* ORES would be used in this way.
In a lot of ways, that's the point of building infrastructure like this --
to empower others to enact their own values.
-Aaron
OK, thanks for the response.
On another note, I recently learned that the purpose of the beta features
extension is to allow extensions to be tested before they can be pushed to
production. This means that after some time (I think, 6 months) we will
have to promote to stable or dismantle the WikiLabels extension. I don't
think promoting is an option as not everyone needs this feature. So I was
wondering if we still want to move the functionality from the gadget to an
extension.
Baha
On Mon, 18 Jan 2016 11:09:38 -0500, Aaron Halfaker
<aaron.halfaker(a)gmail.com> wrote:
> Right now, we build some of that JS based on configuration parameters
> that
> are needed by the WikiLabels service back-end. It seems that it would be
> easiest to generate those files and cache them so that we can serve with
> them extension code.
>
> I'm not sure how much sense this makes long-term. If the answer is "no
> sense -- at all", then I think our next step would be getting a good
> description of which parts of that JS are configured and then working on
> a
> proposal for how we'd change things within the extension. I can do the
> description if we get that far.
>
> -Aaron
>
> On Mon, Jan 18, 2016 at 6:55 AM, Bahodir Mansurov
> <bmansurov(a)wikimedia.org>
> wrote:
>
>> Hello,
>>
>> As I was working on the WikiLabels extension, I had to copy over a lot
>> of
>> CSS and JavaScript files to the extension folder. I suppose it's fine by
>> itself, but I was wondering if those files will continue to be used
>> outside
>> the extension? If so, we should keep them where they are and load them
>> dynamically in the extension (as it's done in the gadget). What do you
>> think?
>>
>> --
>> Baha
>>
>> _______________________________________________
>> AI mailing list
>> AI(a)lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/ai
>>
--
Baha