AI

ai@lists.wikimedia.org

1 participants
265 discussions

Fwd: [Labs-l] [Labs-announce] Some Labs instances rebooting TODAY, 19:00 UTC
by Amir Ladsgroup 06 Aug '16

06 Aug '16

1 0

The Revision Scoring weekly update
by Aaron Halfaker 02 Aug '16

02 Aug '16

Hey, This is the 15th weekly update from revision scoring team that we have sent to this mailing list. *New developments:* - We'll no longer unnecessarily load the models into memory on the web workers[1]. - We can now score multiple models against the same revision ID for (essentially) free[2]. - Our precaching system will take advantage of this to drop load by about 3X[3]. - Update wmflabs deploy repo for new version of ORES[4]. *Documentation & maintenance:* - We completed deployment and maintenance docs for Wiki labels[5], which means we've now got complete docs for our systems[6]. - We implemented basic continuous integration tests for the ORES extension[7]. *Downtime:* - We had a 1 hour long downtime while trying to deploy new code to ores.wikimedia.org[8]. We've filed two critical tasks for making sure we don't make the mistake again[9,10]. 1. https://phabricator.wikimedia.org/T134606 - Score multiple models with the same cached dependencies 2. https://phabricator.wikimedia.org/T139407 - Don't load models into memory of web workers 3. https://phabricator.wikimedia.org/T141376 - Update precached to group requests by model 4. https://phabricator.wikimedia.org/T141377 - Update wmflabs deploy repo for new version of ORES 5. https://phabricator.wikimedia.org/T131768 - Wikilabels deployment docs 6. https://phabricator.wikimedia.org/T106271 - Document maintenance tasks 7. https://phabricator.wikimedia.org/T140455 - CI test for ORES extension 8. https://wikitech.wikimedia.org/wiki/Incident_documentation/20160801-ORES 9. https://phabricator.wikimedia.org/T141823 - Set up password on ORES Beta redis server 10. https://phabricator.wikimedia.org/T141825 - Config beta ORES extension to use the beta ORES service Sincerely, Aaron from the Revision Scoring team

1 0

The Revision Scoring weekly update
by Amir Ladsgroup 26 Jul '16

26 Jul '16

Hey, This is the 14th weekly update from revision scoring team that we have sent to this mailing list. New developments: - We have a new grafana dashboard for requests that the ORES beta feature makes [1] [2] - There is a grafana chart in ORES service showing total number of requests (including requests hitting cache). Showing we had 20 million requests last month! [3] - ORES extension can be set to score edits only in a given set of namespaces. This was useful to reduce failure rate of jobs in ORES extension from 4% to 1% [4] - Scap3 now automatically deploys to canary node for ORES first (it's scb2001.codfw.wmnet). So we can better test deployments beforehand. [5] Maintenance and robustness: - We are removing a redundant index on ores_classification. No harm would be made but we'll save disk space. [6] - In footer of ores.wikimedia.org, we had "Hosted in Wikimedia Labs". It got fixed now. [7] - Cache.php in ORES extension which stores the scores into database has a better structure for handling errors.[8] 1. https://phabricator.wikimedia.org/T141169 2. https://grafana.wikimedia.org/dashboard/db/ores-extension 3. https://grafana.wikimedia.org/dashboard/db/ores 4. https://phabricator.wikimedia.org/T139660 5. https://phabricator.wikimedia.org/T139825 6. https://phabricator.wikimedia.org/T140803 7. https://phabricator.wikimedia.org/T139028 8. https://phabricator.wikimedia.org/T137880 Sincerely, Amir from the Revision Scoring team

1 0

The Revision Scoring weekly update
by Amir Ladsgroup 18 Jul '16

18 Jul '16

Hey, This is the (12 + 1)th [1] weekly update from revision scoring team that we have sent to this mailing list. New developments: - ORES review tool is deployed as a beta feature in Turkish Wikipedia. Now six Wikis have this tool. [2] - CopyPatrol tool soon will show ORES scores if they pass a certain threshold. [3] - We are talking about integration with Detox, feel free to chime in. [4] Maintenance and robustness: - Currently we are dealing with increasing memory pressure on scb nodes. Actions we did to reduce this pressure: - We migrated most of RandomForrest models to GradientBoosting, which will reduce memory pressure greatly without affecting accuracy of models noticeably [5] - It seems there is a memory leak issue with celery. We bypassed that by setting a periodic restart of workers. [6] - We reduced maximum number of precaching requests in order to prevent spikes that might cause memory pressure on other services. [7] - We reduced number of web processes to 2/3. It is still fine. [8] - We finished up the refactor and it will soon goes to the production cluster. [9] - Damaging and goodfaith models had issue in Dutch Wikipedia. Got fixed. [10] - Our metrics collector now sends timed requests. We will have a dashboard in grafana for that soon. [11] - There will be a link to "ORES review tool" page [12] in legend of RecentChanges and Watchlist. [13] - When revscoring fails for any unknown reason, ORES return a proper message now. [14] - We fixed a puppet issue that caused trouble while creating new web instances. Got fixed [15] 1. It's 13. We are not superstitious, just kidding ;) 2. https://phabricator.wikimedia.org/T139992 3. https://phabricator.wikimedia.org/T139009 4. https://phabricator.wikimedia.org/T139007 5. https://phabricator.wikimedia.org/T139963 6. https://phabricator.wikimedia.org/T140020 7. https://gerrit.wikimedia.org/r/299559 8. https://gerrit.wikimedia.org/r/298739 9. https://github.com/wiki-ai/ores/pull/155 10. https://phabricator.wikimedia.org/T140038 11. https://phabricator.wikimedia.org/T137442 12. https://www.mediawiki.org/wiki/ORES_review_tool 13. https://phabricator.wikimedia.org/T140361 14. https://phabricator.wikimedia.org/T140301 15. https://phabricator.wikimedia.org/T140265 Sincerely, Amir from the Revision Scoring team

1 0

The Revision Scoring weekly update
by Amir Ladsgroup 11 Jul '16

11 Jul '16

Hey, This is the 12th weekly update from revision scoring team that we have sent to this mailing list. New developments: - We deployed ORES review tool in Russian and Portuguese Wikipedia! [1] [2] - Basic support for English Wiktionary and Czech Wikipedia is live in labs.[3] Soon production will follow. - Damaging model for Polish Wikipedia is ready and soon be available (then we can enable ORES review tool as well) [4] - ORES extension now highlights the whole row for Enhanced recent changes too. It will be live in two weeks [5] - We built prototype of a small web app to show progress of wikilabels campaigns at https://tools.wmflabs.org/dexbot/tools/wikilabels_stats.php. Feedback is welcome [6] Maintenance and robustness: - If a web node goes down in labs, we get icigna warnings [7] - We re-launched wikilabels campaigns for French and Azeri Wikipedia [8] [9] - We are starting to use tsv2json to load the whole data when adding new campaigns in wikilabels [10] - "Hide/Show good edits" soon will be changed to "Hide/Show probably good edits" in recent changes and watchlists to reflect ORES scores more accurately [11] 1. https://phabricator.wikimedia.org/T139541 2. https://phabricator.wikimedia.org/T139692 3. https://phabricator.wikimedia.org/T139789 4. https://phabricator.wikimedia.org/T139207 5. https://phabricator.wikimedia.org/T139924 6. https://phabricator.wikimedia.org/T139874 7. https://phabricator.wikimedia.org/T134782 8. https://phabricator.wikimedia.org/T107730 9. https://phabricator.wikimedia.org/T139875 10. https://phabricator.wikimedia.org/T139939 11. https://phabricator.wikimedia.org/T139754 Sincerely, Amir from the Revision Scoring team

1 0

The Revision Scoring weekly update
by Amir Ladsgroup 05 Jul '16

05 Jul '16

Hey, This is the 11th weekly update from revision scoring team that we have sent to this mailing list. *New developments:* - ORES review tool as a beta feature is enabled in Dutch Wikipedia. More wikis to come soon this week [1]. - We have basic edit quality model for Czech Wikipedia ready and merged. To be deployed this week [2]. - We also have basic models for English Wiktionary too. This is the second non-Wikipedia project we support after Wikidata [3]. - Thanks to Tar Lócesilion, we have Polish edit quality campaign completed, We are working on building damaging and goodfaith models at the moment [4]. *Maintenance and robustness:* - We decreased our web capacity in order to reduce memory pressure on scb nodes. You should not get any overload error since our capacity is still very high but if you do, please contact us immediately and we will bring it back up [5]. - We improved documentation on ores.wikimedia.org page a little bit. To be deployed this week [6]. We are working on a rather big refactor on ores which will give us performance boost on scoring multiple models at the same time [7] and reduce memory usage [8]. Feel free to chime in and give us feedback [9]. 1. https://phabricator.wikimedia.org/T139432 2. https://phabricator.wikimedia.org/T138885 3. https://phabricator.wikimedia.org/T138630 4. https://phabricator.wikimedia.org/T130269 5. https://phabricator.wikimedia.org/T139177 6. https://phabricator.wikimedia.org/T138089 7. https://phabricator.wikimedia.org/T134606 8. https://phabricator.wikimedia.org/T139407 9. https://phabricator.wikimedia.org/T139408 Sincerely, Amir from the Revision Scoring team.

1 0

The revision scoring weekly update
by Amir Ladsgroup 28 Jun '16

28 Jun '16

Hey all, This is the 10th weekly update (yay!) from revision scoring team that we have sent to this mailing list. We had Wikimania so it was a busy week. Sessions: - Lessons Learned building machine models for Wikidata [1] Video [2] Slides [3] - Hackathon: Supporting new languages in ORES [4] Video [5] New developments: - Thanks to the services team, we now use ChangePropagation for precaching [6] - New version of ORES is deployed in so new edit quality models (svwiki and nowiki) and article quality models (ruwiki) and other new deployments are now live. Labs: [7] Prod [8] - ores in beta cluster is live again with the exact same settings as prod so we can test more carefully [9] - We launched edit quality campaign for Swedish Wikipedia [10] - Thanks to Daniel Zahn, we have monitoring for worker nodes in production cluster [11] - Our precached daemon in labs now support omitting bot edits [12] - Revscoring supports Czech language, allow us enabling basic support for cswiki soon. [13] Maintenance and robustness: - We shipped all of GUI dependencies including Swagger, Wikimedia UI and bootstrap to ORES itself [14] - Fixed Swagger specs for better monitoring by Ops [15] - Fixed puppet issue with worker nodes in labs [16] - Wikilabels campaign in huwiki was broken. Got fixed [17] - URLs without trailing slash won't get redirected and just return the same response [18] - ores-wikimedia-config repo got renamed to ores-wmflabs-config to show the distinction better [19] - We improved some documentation with regard to deployment, in order to prevent down times [20] 1. https://phabricator.wikimedia.org/T138351 2. https://www.youtube.com/watch?v=dPs1evXFZa0 3. https://commons.wikimedia.org/wiki/File:Lessons_learned_building_machine_le… 4. https://phabricator.wikimedia.org/T134628 5. https://www.youtube.com/watch?v=WnM5Ys2-Cb8 6. https://phabricator.wikimedia.org/T138090 7. https://phabricator.wikimedia.org/T137828 8. https://phabricator.wikimedia.org/T138863 9. https://phabricator.wikimedia.org/T138445 10. https://phabricator.wikimedia.org/T131451 11. https://phabricator.wikimedia.org/T138882 12. https://phabricator.wikimedia.org/T137967 13. https://phabricator.wikimedia.org/T131637 14. https://phabricator.wikimedia.org/T137362 15. https://phabricator.wikimedia.org/T137804 16. https://phabricator.wikimedia.org/T137813 17. https://phabricator.wikimedia.org/T138576 18. https://phabricator.wikimedia.org/T138682 19. https://phabricator.wikimedia.org/T138253 20. https://phabricator.wikimedia.org/T138234 Sinceraly Amir from the Revision Scoring team

1 0

ORES going into production
by Aaron Halfaker 22 Jun '16

22 Jun '16

Hey folks, We (The Revision Scoring Team[1]) are happy to announce the deployment of the ORES service[2] in production at a new address: https://ores.wikimedia.org. This will replace the old Wikimedia Labs address soon: https://ores.wmflabs.org. Along with this new location, we are running on more predictable infrastructure that will allow us to increase our uptime and to make the service available to MediaWiki and extensions directly. We've also begun deploying the ORES review tool[3] as a beta feature on Wikidata and Persian Wikipedia in order trial the fully integrated extension. Once enabled, the ORES review tool highlights edits that are likely to be damaging in Special:RecentChanges to help you prioritize your patrolling work. ORES is an experimental technology. We encourage you to take advantage of it but also to be skeptical of the predictions made. Please reach out to us with your questions and concerns. We'll soon begin to deploy the ORES review tool to more wikis. Next up are English, Portuguese, Russian, Dutch and Turkish Wikipedias. We can deploy to these wikis because those communities have completed Wiki labels[4] campaigns that help train ORES' classifiers to differentiate good-faith mistakes from vandalism. If you'd like to get the ORES review tool deployed in your wiki, please reach out to us for help setting up or completing a Wiki labels campaign on your local wiki. Wikimania participants can also attend our workshop[5] during the hackathon to get setting up ORES for your local wiki. Documentation: * https://www.mediawiki.org/wiki/ORES_review_tool * https://www.mediawiki.org/wiki/Extension:ORES * https://meta.wikimedia.org/wiki/Objective_Revision_Evaluation_Service Bugs & feature requests: https://phabricator.wikimedia.org/tag/revision-scoring-as-a-service-backlog/ IRC: #wikimedia-ai[6] 1. https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service#Team 2. https://meta.wikimedia.org/wiki/Objective_Revision_Evaluation_Service 3. https://www.mediawiki.org/wiki/ORES_review_tool 4. https://meta.wikimedia.org/wiki/Wiki_labels 5. https://phabricator.wikimedia.org/T134628 6. https://webchat.freenode.net/?channels=#wikimedia-ai Stay tuned for an update about deprecation of ores.wmflabs.org and announcements of support for new wikis. Please feel free to reach out to us with any questions/ideas. -Aaron

2 1

The revision scoring weekly update
by Aaron Halfaker 20 Jun '16

20 Jun '16

Hey folks, This is the 9th weekly update for the revision scoring team that we have sent to this mailing list. The biggest news is that we've officially deployed the ORES review tool[1] to Persian Wikipedia and Wikidata[2,9]. We'll be posting a more complete announcement to Wikitech-l soon. New developments: - We generalized how the wsgi and celery applications are configured[4] - We fixed the way Special:Contributions is rendered so that the ORES review tool can work better there[6] Maintenance and robustness: - We documented some downtime for the ORES production service[3] - We untangled and simplified our deployment repositories[5] - An issue with Wikilabels accepting too many labels for a single task is fixed[7] - We brought our deployment of ORES in the beta cluster closer to production[8] - We solved an issue with how the ORES extension populates scores[10] Amir will be hosting a workshop for extending ORES support at Wikimania. See https://phabricator.wikimedia.org/T134628 for more details. 1. https://www.mediawiki.org/wiki/ORES_review_tool 2. https://phabricator.wikimedia.org/T130210 3. https://phabricator.wikimedia.org/T138248 4. https://phabricator.wikimedia.org/T137820 5. https://phabricator.wikimedia.org/T137819 6. https://phabricator.wikimedia.org/T122537 7. https://phabricator.wikimedia.org/T131997 8. https://phabricator.wikimedia.org/T137821 9. https://phabricator.wikimedia.org/T130212 10. https://phabricator.wikimedia.org/T137854 -Aaron

1 0

Re: [AI] [Wikidata] ORES live as a beta feature on Wikidata
by Pine W 16 Jun '16

16 Jun '16

Lydia and Amir FTW. (There seems to be a pattern of good news from the Wikidata and ORES teams. I wonder if Team Practices could facilitate the replication of some of their methods!) Onward ho, Pine On Jun 16, 2016 10:14, "Lydia Pintscher" <Lydia.Pintscher(a)wikimedia.de> wrote: Hey folks :) Amir and other have worked hard over the past months to bring ORES to Wikidata. The goal is to use machine learning to make it easier to spot potentially bad edits. ORES is now available as a beta feature on Wikidata. Once you have enabled it you can see some edits in recent changes and watchlist will show up in a different color or have a little r in front of them. These edits are judged as potentially bad and should probably get more review. In your preferences you can adjust how harsh ORES should judge. You can also filter your watchlist/recent changes to only show potentially bad edits. Patrolled edits won't be shown as potentially bad. This should be a huge step towards making it easier to find and fight vandalism on Wikidata. Cheers Lydia -- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

AI