The fuck AI request, be transparent
On Nov 30, 2016 2:00 PM, <ai-request(a)lists.wikimedia.org> wrote:
> Send AI mailing list submissions to
> ai(a)lists.wikimedia.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.wikimedia.org/mailman/listinfo/ai
> or, via email, send a message with subject or body 'help' to
> ai-request(a)lists.wikimedia.org
>
> You can reach the person managing the list at
> ai-owner(a)lists.wikimedia.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of AI digest..."
>
>
> Today's Topics:
>
> 1. The Revision scoring weekly update (Aaron Halfaker)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 29 Nov 2016 18:34:11 -0500
> From: Aaron Halfaker <aaron.halfaker(a)gmail.com>
> To: Application of Artificial Intelligence and other advanced
> computing strategies to Wikimedia Projects <ai(a)lists.wikimedia.org
> >,
> Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
> Subject: [AI] The Revision scoring weekly update
> Message-ID:
> <CANQe2T_GAqzZYraq7vm+WcSZi+x-1p7K_HP1PTvYgjEUY4_OrQ@mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hey,
>
> This is the 30th and 31st weekly update from the revision scoring team that
> we have sent to this mailing list. We accidentally skipped a week again.
>
> *New development:*
>
> - We added a new "lowest" sensitivity level to ORES review tool. This
> new sensistivity level will only flag edits that ORES is very confident
> are
> actually damaging[1].
>
>
> - We applied the MediaWiki standard color palette to Wikilabels[2]
>
>
> - We generated a manually censored public dataset of
> spam/vandalism/attack pages[3]. This will help others to develop spam,
> vandalism and attack page detection models. See the publication of the
> dataset[4].
>
>
> - We've implement color-based confidence reporting for ORES damage
> detection[5]
>
>
> *Maintenance and robustness:*
>
> - We updated the version of OOjs-UI that gets bundled with Wiki
> labels[6] and moved the static assets to a new repositiory[7]
>
>
> - We fixed an issue in the recscoring library[8] that caused ORES to
> return invalid JSON and rendered the UI useless[9].
>
>
> *Communications:*
>
> - We gave a 3 minute presentation on the state of ORES to Victoria
> Coleman, the WMF's new CTO[10].
>
>
> - We performed a basic analysis of Wikipedia article quality trends
> using the dataset we released a few weeks ago[11]. We'll have a more
> substantial analysis soon.
>
>
> - We made a post on the ORES review tool talk page[12,13] detailing how
> we plan to incorporate a new filtering strategy into the ORES review
> tool.
> Please join the discussion there.
>
>
> 1. https://phabricator.wikimedia.org/T150224 -- Add "Lowest" ORES
> sensitivity for fpr=0.1
> 2. https://phabricator.wikimedia.org/T151119 -- Apply ui standardization
> color palette to Wikilabels
> 3. https://phabricator.wikimedia.org/T150307 -- Create manually vetted
> dataset of spam/vandalism/attack pages
> 4. https://dx.doi.org/10.6084/m9.figshare.4245035
> 5. https://phabricator.wikimedia.org/T144922 -- Visually report damaging
> confidence
> 6. https://phabricator.wikimedia.org/T151222 -- Update bundled OOJS-ui
> with
> Wikilabels
> 7. https://github.com/wiki-ai/flask-oojsui
> 8. https://phabricator.wikimedia.org/T150961 -- ORES ui is broken (text
> field disabled)
> 9. https://github.com/wiki-ai/ores/issues/177
> 10. https://phabricator.wikimedia.org/T150544 -- ORES (a 2-3 minute
> presentation)
> 11. https://phabricator.wikimedia.org/T151214 -- Basic analysis of
> Wikipedia quality using monthly predictions
> 12. https://phabricator.wikimedia.org/T150858 -- Post about ORES review
> tool including ERI filters
> 13. https://www.mediawiki.org/wiki/Topic:Tflhjj5x1numzg67
>
> Sincerely,
> Aaron from the Revision Scoring team
>
Hey,
This is the 30th and 31st weekly update from the revision scoring team that
we have sent to this mailing list. We accidentally skipped a week again.
*New development:*
- We added a new "lowest" sensitivity level to ORES review tool. This
new sensistivity level will only flag edits that ORES is very confident are
actually damaging[1].
- We applied the MediaWiki standard color palette to Wikilabels[2]
- We generated a manually censored public dataset of
spam/vandalism/attack pages[3]. This will help others to develop spam,
vandalism and attack page detection models. See the publication of the
dataset[4].
- We've implement color-based confidence reporting for ORES damage
detection[5]
*Maintenance and robustness:*
- We updated the version of OOjs-UI that gets bundled with Wiki
labels[6] and moved the static assets to a new repositiory[7]
- We fixed an issue in the recscoring library[8] that caused ORES to
return invalid JSON and rendered the UI useless[9].
*Communications:*
- We gave a 3 minute presentation on the state of ORES to Victoria
Coleman, the WMF's new CTO[10].
- We performed a basic analysis of Wikipedia article quality trends
using the dataset we released a few weeks ago[11]. We'll have a more
substantial analysis soon.
- We made a post on the ORES review tool talk page[12,13] detailing how
we plan to incorporate a new filtering strategy into the ORES review tool.
Please join the discussion there.
1. https://phabricator.wikimedia.org/T150224 -- Add "Lowest" ORES
sensitivity for fpr=0.1
2. https://phabricator.wikimedia.org/T151119 -- Apply ui standardization
color palette to Wikilabels
3. https://phabricator.wikimedia.org/T150307 -- Create manually vetted
dataset of spam/vandalism/attack pages
4. https://dx.doi.org/10.6084/m9.figshare.4245035
5. https://phabricator.wikimedia.org/T144922 -- Visually report damaging
confidence
6. https://phabricator.wikimedia.org/T151222 -- Update bundled OOJS-ui with
Wikilabels
7. https://github.com/wiki-ai/flask-oojsui
8. https://phabricator.wikimedia.org/T150961 -- ORES ui is broken (text
field disabled)
9. https://github.com/wiki-ai/ores/issues/177
10. https://phabricator.wikimedia.org/T150544 -- ORES (a 2-3 minute
presentation)
11. https://phabricator.wikimedia.org/T151214 -- Basic analysis of
Wikipedia quality using monthly predictions
12. https://phabricator.wikimedia.org/T150858 -- Post about ORES review
tool including ERI filters
13. https://www.mediawiki.org/wiki/Topic:Tflhjj5x1numzg67
Sincerely,
Aaron from the Revision Scoring team
Hey,
With merge of 320328 [1] and 320341, two major changes will come to ORES
review tool:
1- You will see one more option in ORES sensitivity called "Lowest". It
means if you choose it, it only flags edit that are very likely to be
vandalism.
2- Coloring of rows will be completely different. You will see several
colors instead of one and as confidence of ORES grows, the colors will tend
to be more noticeable. It goes without saying that you can change these
colors in your own css. I put a screenshot in [3] and you can test it in
https://en.wikipedia.beta.wmflabs.org or https://mw-revscoring.wmflabs.org
Feedback is always welcome
[1]: https://gerrit.wikimedia.org/r/#/c/320328/
[2]: https://gerrit.wikimedia.org/r/#/c/320341/
[3]: https://phabricator.wikimedia.org/T144922#2824696
Best
--
Amir Sarabadani Tafreshi
Software Engineer (contractor)
-------------------------------------
Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
http://wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
Hey folks,
I'm your friendly facilitator for who forgot that today was the last day to
gather discussion on a set of topics of the Dev Summit. I might be a bit
biased, but I think they are all pretty interesting, so I'm reaching out
with a quick overview to see if I can spur some interest from ya'll. Check
'em out:
- https://phabricator.wikimedia.org/T149373 -- Evaluating the user
experience of AI systems
- https://phabricator.wikimedia.org/T147710 -- Building an AI wishlist &
working groups for Wikimedia Projects
- https://phabricator.wikimedia.org/T148690 -- Where to surface AI in
Wikimedia Projects
- https://phabricator.wikimedia.org/T147929 -- Algorithmic dangers and
transparency -- Best practices
- https://phabricator.wikimedia.org/T149666 -- Next steps for machine
translation
If you're interested, please drop a note or a token in the task. BTW, you
don't have to physically attend the dev summit in order to participate.
I'll make sure that IRC and Etherpad are shared with all remote attendees
who want to attend the sessions I'm helping to organize. I've heard that
there will be additional facilities for remote attendees (maybe a youtube
stream!?) this year, but I can't confirm yet.
-Aaron
Hey,
This is the 29th weekly update from revision scoring team that we have sent
to this mailing list.
Deployments:
- We deployed logging changes to ORES that will reduce the verbosity[1]
- We also deployed revscoring 1.3.0 and new models built with it to WMF
labs[2]. This won't change anything important from a user-perspective, but
it paves the way for developing new modeling strategies.
Maintenance and robustness:
- We fixed puppet so that log file directories are also created on the
celery worker nodes (affects wmflabs)[3]
- We fixed an issue with our recall_at_fpr metrics which was incorrectly
defined and implemented a recall_at_precision metric to take its place[4]
New development:
- We've made a lot of progress on modeling sentences and have just
started experimenting with a sentence model from featured articles[5]
- We're reviewing a dataset of spam/vandalism/attack new page creations
for public release[6]. This dataset will help our collaborators work with
us on modeling the quality of drafts and supporting new page triage.
1. https://phabricator.wikimedia.org/T149730 -- Deploy logging changes to
ORES
2. https://phabricator.wikimedia.org/T150447 -- Deploy revscoring 1.3.0 and
updated editquality and wikiclass to wmflabs
3. https://phabricator.wikimedia.org/T149925 -- /srv/log/ores/ not created
on worker nodes
4. https://phabricator.wikimedia.org/T149825 -- Implement recall at
precision (and fix FPR metrics)
5. https://phabricator.wikimedia.org/T148867 -- Implement sentences
datascources & experiment with normalization.
6. https://phabricator.wikimedia.org/T150307 -- Create manually vetted
dataset of spam/vandalism/attack pages
Sincerely,
Aaron from the Revision Scoring team