AI September 2016

ai@lists.wikimedia.org

7 participants
17 discussions

Re: [AI] ORES article scores
by Pine W 16 Sep '16

16 Sep '16

Hi Joe, The particular use case that I have in mind is that a reader, or potential contributor, wants to know the quality of a particular article. (Note that this is a slightly different use case from sorting many articles by their ORES scores, or comparing their ORES scores to the importance and/or manually assigned article quality grade.) ORES scores are likely to be more frequently updated than manual article quality grades, and I feel that this information would be useful to readers and/or potential contributors. Having the ORES score templates on talk pages, placed there by a bot and later refreshed weekly, would be adequate for the use case that I have in mind. Note that https://upload.wikimedia.org/wikipedia/commons/6/6a/ Wikimedia_Foundation_and_Reboot_New_Readers_Research_-_ Nigeria_%26_India_Highlights_-_July_2016.pdf suggests that WMF Product "Add an indicator of quality icon (akin to the signal strength icon for mobile networks) on each article, based on research and analysis on article contributions and edits." I am thinking that ORES scores, and/or manual article quality grades, could serve this purpose. English Wikipedia already displays some article grades on the front pages of articles via the GA or FA icons, and my impression is that the New Readers Research suggests making some kind of quality indicator more prominent and on all Wikipedia wikis. This would involve work that's beyond the scope of what I have in mind, but I think that it's worth noting for potential future development. Abbey may wish to comment on this. Pine On Fri, Sep 16, 2016 at 2:06 PM, Joe Matazzoni <jmatazzoni(a)wikimedia.org> wrote: > The project we’re working on with ORES (Edit Review Improvements > <https://www.mediawiki.org/wiki/Edit_Review_Improvements> ) is concerned > only with the Damaging and Good Faith scores at this time. So I don’t know > too much about the ORES article scores. I believe they have a lot of > potential, but I’m a little unclear on the general goal of this proposal. > > My sense of the ORES article scores is that they’re most useful for people > who are not (yet) looking at the article. E.g., the scores might be used to > review AfC backlogs and surface the pages that are ready for re-review. Or > they might be used to help reviewers engaged in finding and promoting > Featured Articles. They might conceivably have a place in search results > some day, as additional data for readers. > > My guess is that these scores are based at this time on formal > characteristics of the article — its length, its conformance with > formatting standards, whether it includes photos or multimedia, whether it > has references, etc. These formal characteristics would for the most part, > I’d think, be obvious to someone already on the article page. > > But ORES does not know if a subject is notable, for example. Nor do t > believe it would know if an article were factually correct, among other > things. > > So, again, what is your goal? Is there a Phabricator task? Good luck! > > Joe > > > > _____________________ > > *Joe Matazzoni* > Product Manager, Collaboration > Wikimedia Foundation, San Francisco > mobile 202.744.7910 > jmatazzoni(a)wikimedia.org > *"Imagine a world in which every single human being can freely share in > the sum of all knowledge." * > > > > > On Sep 15, 2016, at 8:57 PM, Pine W <wiki.pine(a)gmail.com> wrote: > > Joe, any comment about this? > > Toby and Amir, would it make sense to integrate ORES scores into > https://meta.wikimedia.org/wiki/Community_Tech/PageAssessments, or have > some other kind of relation between the ability to pull ORES scores from > the MW API and the PageAssessments API? Or does it make sense to have them > be separate but working in parallel? I'm an API novice, but the > functionality of PageAssessments reminds me of what we're discussing here, > which is why I bring it up. > > Thanks, > > Pine > > On Wed, Aug 31, 2016 at 2:48 PM, Toby Negrin <tnegrin(a)wikimedia.org> > wrote: > >> Hi Pine -- thanks for this. I'm looping in Joe M. from the Collaboration >> team who can speak to his team's thoughts on ORES scores. >> >> It seems likely that my team will devote some resources to making ORES (+ >> the page view API) accessible via the MW API next quarter which will help >> integration efforts. Our goals go up on Friday and I just got out of a >> meeting where we agreed to do this and it's unlikely this will change in 2 >> days. >> >> As far as New Readers though, I suspect that it's a bit too abstract as >> of yet. I think we should start with Old Readers :) >> >> -Toby >> >> On Wed, Aug 31, 2016 at 2:37 PM, Pine W <wiki.pine(a)gmail.com> wrote: >> >>> Just to note that https://upload.wikimedia.org/w >>> ikipedia/commons/6/6a/Wikimedia_Foundation_and_Reboot_New_Re >>> aders_Research_-_Nigeria_%26_India_Highlights_-_July_2016.pdf suggests >>> that WMF Product "Add an indicator of quality icon (akin to the signal >>> strength icon for mobile networks) on each article, based on research and >>> analysis on article contributions and edits." This is similar to what I >>> have in mind for talk pages with ORES scores. Adding indicators to the >>> front page of articles would likely require an RfC which may be more >>> involved than an RfB to add the score to talk pages. Still, I can see how >>> having an ORES score, or the Wikipedia Quality score (at least on ENWP) >>> indicated by an icon at the top of article pages would be useful, >>> especially if readers can intuitively understand what the indicator is >>> telling them without needing to look at documentation. >>> >>> I think it might be best to start with ORES scores on talk pages. That >>> fills the need that I have in mind. If someone wants to extend this to >>> having article quality indicators on the front page of articles, I'd be >>> happy to collaborate with them. >>> >>> Pine >>> >>> On Tue, Aug 16, 2016 at 7:32 AM, Pine W <wiki.pine(a)gmail.com> wrote: >>> >>>> Thanks. I will add this to my agenda for later this year. >>>> >>>> Pine >>>> >>>> On Aug 16, 2016 01:48, "Amir Ladsgroup" <ladsgroup(a)gmail.com> wrote: >>>> >>>>> Oh, Sorry I missed the question. Yes, it's fairly easy :) >>>>> >>>>> Best >>>>> >>>>> On Tue, Aug 16, 2016 at 12:15 PM Pine W <wiki.pine(a)gmail.com> wrote: >>>>> >>>>>> Pinging again to repeat my question from my previous email. >>>>>> >>>>>> Thanks, >>>>>> Pine >>>>>> >>>>>> On Aug 9, 2016 16:04, "Pine W" <wiki.pine(a)gmail.com> wrote: >>>>>> >>>>>>> I'd like to work on this if I had the time to learn and to do it. >>>>>>> Maybe I will in early September or sometime later this year. I've put it on >>>>>>> my to do list. This would be my first bot; do you think that it's a good >>>>>>> project for a newbie bot builder? >>>>>>> >>>>>>> Pine >>>>>>> >>>>>>> On Sat, Aug 6, 2016 at 11:16 AM, Amir Ladsgroup <ladsgroup(a)gmail.com >>>>>>> > wrote: >>>>>>> >>>>>>>> Building that is easy, you just need someone to add these >>>>>>>> functionalities to the templates and a bot to run it for the first time. >>>>>>>> >>>>>>>> Any volunteers? >>>>>>>> >>>>>>>> Best >>>>>>>> >>>>>>>> On Mon, Aug 1, 2016 at 9:37 PM Pine W <wiki.pine(a)gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi Amir, >>>>>>>>> >>>>>>>>> I think that I'm following better now. >>>>>>>>> >>>>>>>>> The extension option doesn't sound workable for what I have in >>>>>>>>> mind. I'd like the score to be visible without needing to be logged in, >>>>>>>>> which I believe means that the feature can't be conditional on having >>>>>>>>> access to beta features. >>>>>>>>> >>>>>>>>> The solution to click on a talk page link for an updated score >>>>>>>>> sounds more workable. It would be best if the score refreshed automatically >>>>>>>>> every time someone viewed the talk page for the purpose that I have in >>>>>>>>> mind, but I don't know what kind of performance penalty and computing >>>>>>>>> resources that would require. If either of those are nontrivial, then >>>>>>>>> having a template on the talk page that says something like "Score last >>>>>>>>> refreshed on TIME DATE. There have been X revisions since that time. Click >>>>>>>>> here to update the score" would be good for the use case that I have in >>>>>>>>> mind. >>>>>>>>> >>>>>>>>> Pine >>>>>>>>> >>>>>>>>> On Mon, Aug 1, 2016 at 1:48 AM, Amir Ladsgroup < >>>>>>>>> ladsgroup(a)gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hey, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sun, Jul 31, 2016 at 8:16 AM Pine W <wiki.pine(a)gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Amir, >>>>>>>>>>> >>>>>>>>>>> Thanks for waiting. As I understand it, we have these options >>>>>>>>>>> for making the ORES scores visible on article talk pages: >>>>>>>>>>> >>>>>>>>>>> 1. A gadget >>>>>>>>>>> 2. An addition to the ORES extension >>>>>>>>>>> 3. A bot >>>>>>>>>>> >>>>>>>>>>> Is that correct? >>>>>>>>>>> >>>>>>>>>> Yes >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> WP:WPP refers to WikiProject Philosophy; did you man something >>>>>>>>>>> else or am I misunderstanding the abbreviation? >>>>>>>>>>> >>>>>>>>>>> My bad, I meant something like this: >>>>>>>>>> https://en.wikipedia.org/wiki/Template:WikiProject_Physics >>>>>>>>>> >>>>>>>>>>> I'm thinking that a gadget might be a workable solution, but my >>>>>>>>>>> preference would be to have the score displayed on all article talk pages >>>>>>>>>>> automatically for everyone to see including readers who are not logged in. >>>>>>>>>>> A bot that posted scores to talk pages and updated the scores on a weekly >>>>>>>>>>> basis would be good for that purpose. >>>>>>>>>>> >>>>>>>>>>> Both are possible solutions but as for bots. Updating the score >>>>>>>>>> for all articles of a wikiproject (e.g. Wikiproject medicine has 35K >>>>>>>>>> articles) on weekly basis is not efficient. That bot needs to do 35K edits >>>>>>>>>> every week and it might make some people mad. We can have an initial >>>>>>>>>> release and update it on demand (like adding a link to the score in the >>>>>>>>>> template. If someone wants that score gets updated. Just clicks on the link >>>>>>>>>> and the bot update it) >>>>>>>>>> >>>>>>>>>>> I'm unclear on how an addition to the ORES extension would work; >>>>>>>>>>> can you explain further what that option would involve? >>>>>>>>>>> >>>>>>>>>> Adding it to the ores extension will make things much easier. You >>>>>>>>>> just need to enable it in your beta features and then in the talk page or >>>>>>>>>> action=info you will see the score. Implementing it is difficult due to >>>>>>>>>> storage issues for the database and we might need to get operations >>>>>>>>>> approval before continuing. >>>>>>>>>> >>>>>>>>>> Best >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Pine >>>>>>>>>>> >>>>>>>>>>> On Sun, Jul 24, 2016 at 11:29 PM, Pine W <wiki.pine(a)gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Amir, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for getting back to me. My mind is a little full at the >>>>>>>>>>>> moment, and I have some deadlines this week, so it may take me a little >>>>>>>>>>>> while to get back to you. But I'm flagging this conversation for follow up, >>>>>>>>>>>> hopefully by the end of this week or early next week. >>>>>>>>>>>> >>>>>>>>>>>> Thanks for your patience, >>>>>>>>>>>> Pine >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Jul 22, 2016 at 9:59 PM, Amir Ladsgroup < >>>>>>>>>>>> ladsgroup(a)gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hey, >>>>>>>>>>>>> ORES has "article quality" models for English, Russian and >>>>>>>>>>>>> French Wikipedia. There is a very short introduction in ORES >>>>>>>>>>>>> <https://meta.wikimedia.org/wiki/Objective_Revision_Evaluation_Service#Artic…> >>>>>>>>>>>>> description page. >>>>>>>>>>>>> But if I want to explain it my way. It'll give you a >>>>>>>>>>>>> prediction of class of the article (A, B, C, FA, GA, stub, start) for a >>>>>>>>>>>>> given revision. For example for this revision >>>>>>>>>>>>> <https://en.wikipedia.org/w/index.php?title=Alan_Turing&oldid=730400015> ORES >>>>>>>>>>>>> predicts >>>>>>>>>>>>> <https://ores.wikimedia.org/v2/scores/enwiki/?models=wp10&revids=730400015> >>>>>>>>>>>>> that it's GA (which it is). >>>>>>>>>>>>> The biggest use case for now is that people produce reports >>>>>>>>>>>>> of articles needing re >>>>>>>>>>>>> <https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service/Stor…> >>>>>>>>>>>>> assessment >>>>>>>>>>>>> <https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service/Stor…> for >>>>>>>>>>>>> wikiprojects >>>>>>>>>>>>> <https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service/Stor…>. >>>>>>>>>>>>> The other use case is list of popular articles that ORES >>>>>>>>>>>>> thinks their quality is low >>>>>>>>>>>>> <https://en.wikipedia.org/wiki/User:DataflowBot/output/Popular_low_quality_a…> >>>>>>>>>>>>> . >>>>>>>>>>>>> >>>>>>>>>>>>> Your use case about ORES prediction is valid the second part >>>>>>>>>>>>> is a little bit complicated though. For the first part, we have either >>>>>>>>>>>>> option of a gadget or adding it to the ORES extension. If it's very >>>>>>>>>>>>> important to let everyone see the scores, we can write a bot to update >>>>>>>>>>>>> reports weekly in English Wikipedia for articles in a certain Wikiproject >>>>>>>>>>>>> and we use these data in the WPP templates. Which one do you think suits >>>>>>>>>>>>> English Wikipedia better? >>>>>>>>>>>>> >>>>>>>>>>>>> Best >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Sat, Jul 23, 2016 at 2:57 AM Pine W <wiki.pine(a)gmail.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi artificial intelligence people, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Are there baseline ORES scores available for articles? If so, >>>>>>>>>>>>>> I'm thinking that it would be interesting to have the option to add a >>>>>>>>>>>>>> template to an article's talk page that shows its ORES score as well as its >>>>>>>>>>>>>> ORES percentile rank on a particular wiki. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Pine >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> AI mailing list >>>>>>>>>>>>>> AI(a)lists.wikimedia.org >>>>>>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> AI mailing list >>>>>>>>>>>>> AI(a)lists.wikimedia.org >>>>>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> AI mailing list >>>>>>>>>>> AI(a)lists.wikimedia.org >>>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> AI mailing list >>>>>>>>>> AI(a)lists.wikimedia.org >>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>>>>>> >>>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> AI mailing list >>>>>>>>> AI(a)lists.wikimedia.org >>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> AI mailing list >>>>>>>> AI(a)lists.wikimedia.org >>>>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>> AI mailing list >>>>>> AI(a)lists.wikimedia.org >>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>> >>>>> >>>>> _______________________________________________ >>>>> AI mailing list >>>>> AI(a)lists.wikimedia.org >>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>> >>>>> >>> >> > >

1 0

Re: [AI] ORES article scores
by Pine W 15 Sep '16

15 Sep '16

Joe, any comment about this? Toby and Amir, would it make sense to integrate ORES scores into https://meta.wikimedia.org/wiki/Community_Tech/PageAssessments, or have some other kind of relation between the ability to pull ORES scores from the MW API and the PageAssessments API? Or does it make sense to have them be separate but working in parallel? I'm an API novice, but the functionality of PageAssessments reminds me of what we're discussing here, which is why I bring it up. Thanks, Pine On Wed, Aug 31, 2016 at 2:48 PM, Toby Negrin <tnegrin(a)wikimedia.org> wrote: > Hi Pine -- thanks for this. I'm looping in Joe M. from the Collaboration > team who can speak to his team's thoughts on ORES scores. > > It seems likely that my team will devote some resources to making ORES (+ > the page view API) accessible via the MW API next quarter which will help > integration efforts. Our goals go up on Friday and I just got out of a > meeting where we agreed to do this and it's unlikely this will change in 2 > days. > > As far as New Readers though, I suspect that it's a bit too abstract as of > yet. I think we should start with Old Readers :) > > -Toby > > On Wed, Aug 31, 2016 at 2:37 PM, Pine W <wiki.pine(a)gmail.com> wrote: > >> Just to note that https://upload.wikimedia.org/w >> ikipedia/commons/6/6a/Wikimedia_Foundation_and_Reboot_New_ >> Readers_Research_-_Nigeria_%26_India_Highlights_-_July_2016.pdf suggests >> that WMF Product "Add an indicator of quality icon (akin to the signal >> strength icon for mobile networks) on each article, based on research and >> analysis on article contributions and edits." This is similar to what I >> have in mind for talk pages with ORES scores. Adding indicators to the >> front page of articles would likely require an RfC which may be more >> involved than an RfB to add the score to talk pages. Still, I can see how >> having an ORES score, or the Wikipedia Quality score (at least on ENWP) >> indicated by an icon at the top of article pages would be useful, >> especially if readers can intuitively understand what the indicator is >> telling them without needing to look at documentation. >> >> I think it might be best to start with ORES scores on talk pages. That >> fills the need that I have in mind. If someone wants to extend this to >> having article quality indicators on the front page of articles, I'd be >> happy to collaborate with them. >> >> Pine >> >> On Tue, Aug 16, 2016 at 7:32 AM, Pine W <wiki.pine(a)gmail.com> wrote: >> >>> Thanks. I will add this to my agenda for later this year. >>> >>> Pine >>> >>> On Aug 16, 2016 01:48, "Amir Ladsgroup" <ladsgroup(a)gmail.com> wrote: >>> >>>> Oh, Sorry I missed the question. Yes, it's fairly easy :) >>>> >>>> Best >>>> >>>> On Tue, Aug 16, 2016 at 12:15 PM Pine W <wiki.pine(a)gmail.com> wrote: >>>> >>>>> Pinging again to repeat my question from my previous email. >>>>> >>>>> Thanks, >>>>> Pine >>>>> >>>>> On Aug 9, 2016 16:04, "Pine W" <wiki.pine(a)gmail.com> wrote: >>>>> >>>>>> I'd like to work on this if I had the time to learn and to do it. >>>>>> Maybe I will in early September or sometime later this year. I've put it on >>>>>> my to do list. This would be my first bot; do you think that it's a good >>>>>> project for a newbie bot builder? >>>>>> >>>>>> Pine >>>>>> >>>>>> On Sat, Aug 6, 2016 at 11:16 AM, Amir Ladsgroup <ladsgroup(a)gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Building that is easy, you just need someone to add these >>>>>>> functionalities to the templates and a bot to run it for the first time. >>>>>>> >>>>>>> Any volunteers? >>>>>>> >>>>>>> Best >>>>>>> >>>>>>> On Mon, Aug 1, 2016 at 9:37 PM Pine W <wiki.pine(a)gmail.com> wrote: >>>>>>> >>>>>>>> Hi Amir, >>>>>>>> >>>>>>>> I think that I'm following better now. >>>>>>>> >>>>>>>> The extension option doesn't sound workable for what I have in >>>>>>>> mind. I'd like the score to be visible without needing to be logged in, >>>>>>>> which I believe means that the feature can't be conditional on having >>>>>>>> access to beta features. >>>>>>>> >>>>>>>> The solution to click on a talk page link for an updated score >>>>>>>> sounds more workable. It would be best if the score refreshed automatically >>>>>>>> every time someone viewed the talk page for the purpose that I have in >>>>>>>> mind, but I don't know what kind of performance penalty and computing >>>>>>>> resources that would require. If either of those are nontrivial, then >>>>>>>> having a template on the talk page that says something like "Score last >>>>>>>> refreshed on TIME DATE. There have been X revisions since that time. Click >>>>>>>> here to update the score" would be good for the use case that I have in >>>>>>>> mind. >>>>>>>> >>>>>>>> Pine >>>>>>>> >>>>>>>> On Mon, Aug 1, 2016 at 1:48 AM, Amir Ladsgroup <ladsgroup(a)gmail.com >>>>>>>> > wrote: >>>>>>>> >>>>>>>>> Hey, >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Jul 31, 2016 at 8:16 AM Pine W <wiki.pine(a)gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Amir, >>>>>>>>>> >>>>>>>>>> Thanks for waiting. As I understand it, we have these options for >>>>>>>>>> making the ORES scores visible on article talk pages: >>>>>>>>>> >>>>>>>>>> 1. A gadget >>>>>>>>>> 2. An addition to the ORES extension >>>>>>>>>> 3. A bot >>>>>>>>>> >>>>>>>>>> Is that correct? >>>>>>>>>> >>>>>>>>> Yes >>>>>>>>> >>>>>>>>>> >>>>>>>>>> WP:WPP refers to WikiProject Philosophy; did you man something >>>>>>>>>> else or am I misunderstanding the abbreviation? >>>>>>>>>> >>>>>>>>>> My bad, I meant something like this: >>>>>>>>> https://en.wikipedia.org/wiki/Template:WikiProject_Physics >>>>>>>>> >>>>>>>>>> I'm thinking that a gadget might be a workable solution, but my >>>>>>>>>> preference would be to have the score displayed on all article talk pages >>>>>>>>>> automatically for everyone to see including readers who are not logged in. >>>>>>>>>> A bot that posted scores to talk pages and updated the scores on a weekly >>>>>>>>>> basis would be good for that purpose. >>>>>>>>>> >>>>>>>>>> Both are possible solutions but as for bots. Updating the score >>>>>>>>> for all articles of a wikiproject (e.g. Wikiproject medicine has 35K >>>>>>>>> articles) on weekly basis is not efficient. That bot needs to do 35K edits >>>>>>>>> every week and it might make some people mad. We can have an initial >>>>>>>>> release and update it on demand (like adding a link to the score in the >>>>>>>>> template. If someone wants that score gets updated. Just clicks on the link >>>>>>>>> and the bot update it) >>>>>>>>> >>>>>>>>>> I'm unclear on how an addition to the ORES extension would work; >>>>>>>>>> can you explain further what that option would involve? >>>>>>>>>> >>>>>>>>> Adding it to the ores extension will make things much easier. You >>>>>>>>> just need to enable it in your beta features and then in the talk page or >>>>>>>>> action=info you will see the score. Implementing it is difficult due to >>>>>>>>> storage issues for the database and we might need to get operations >>>>>>>>> approval before continuing. >>>>>>>>> >>>>>>>>> Best >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Pine >>>>>>>>>> >>>>>>>>>> On Sun, Jul 24, 2016 at 11:29 PM, Pine W <wiki.pine(a)gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Amir, >>>>>>>>>>> >>>>>>>>>>> Thanks for getting back to me. My mind is a little full at the >>>>>>>>>>> moment, and I have some deadlines this week, so it may take me a little >>>>>>>>>>> while to get back to you. But I'm flagging this conversation for follow up, >>>>>>>>>>> hopefully by the end of this week or early next week. >>>>>>>>>>> >>>>>>>>>>> Thanks for your patience, >>>>>>>>>>> Pine >>>>>>>>>>> >>>>>>>>>>> On Fri, Jul 22, 2016 at 9:59 PM, Amir Ladsgroup < >>>>>>>>>>> ladsgroup(a)gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hey, >>>>>>>>>>>> ORES has "article quality" models for English, Russian and >>>>>>>>>>>> French Wikipedia. There is a very short introduction in ORES >>>>>>>>>>>> <https://meta.wikimedia.org/wiki/Objective_Revision_Evaluation_Service#Artic…> >>>>>>>>>>>> description page. >>>>>>>>>>>> But if I want to explain it my way. It'll give you a prediction >>>>>>>>>>>> of class of the article (A, B, C, FA, GA, stub, start) for a given >>>>>>>>>>>> revision. For example for this revision >>>>>>>>>>>> <https://en.wikipedia.org/w/index.php?title=Alan_Turing&oldid=730400015> ORES >>>>>>>>>>>> predicts >>>>>>>>>>>> <https://ores.wikimedia.org/v2/scores/enwiki/?models=wp10&revids=730400015> >>>>>>>>>>>> that it's GA (which it is). >>>>>>>>>>>> The biggest use case for now is that people produce reports of >>>>>>>>>>>> articles needing re >>>>>>>>>>>> <https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service/Stor…> >>>>>>>>>>>> assessment >>>>>>>>>>>> <https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service/Stor…> for >>>>>>>>>>>> wikiprojects >>>>>>>>>>>> <https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service/Stor…>. >>>>>>>>>>>> The other use case is list of popular articles that ORES >>>>>>>>>>>> thinks their quality is low >>>>>>>>>>>> <https://en.wikipedia.org/wiki/User:DataflowBot/output/Popular_low_quality_a…> >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>>>> Your use case about ORES prediction is valid the second part is >>>>>>>>>>>> a little bit complicated though. For the first part, we have either option >>>>>>>>>>>> of a gadget or adding it to the ORES extension. If it's very important to >>>>>>>>>>>> let everyone see the scores, we can write a bot to update reports weekly in >>>>>>>>>>>> English Wikipedia for articles in a certain Wikiproject and we use these >>>>>>>>>>>> data in the WPP templates. Which one do you think suits English Wikipedia >>>>>>>>>>>> better? >>>>>>>>>>>> >>>>>>>>>>>> Best >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Jul 23, 2016 at 2:57 AM Pine W <wiki.pine(a)gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi artificial intelligence people, >>>>>>>>>>>>> >>>>>>>>>>>>> Are there baseline ORES scores available for articles? If so, >>>>>>>>>>>>> I'm thinking that it would be interesting to have the option to add a >>>>>>>>>>>>> template to an article's talk page that shows its ORES score as well as its >>>>>>>>>>>>> ORES percentile rank on a particular wiki. >>>>>>>>>>>>> >>>>>>>>>>>>> Pine >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> AI mailing list >>>>>>>>>>>>> AI(a)lists.wikimedia.org >>>>>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> AI mailing list >>>>>>>>>>>> AI(a)lists.wikimedia.org >>>>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> AI mailing list >>>>>>>>>> AI(a)lists.wikimedia.org >>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> AI mailing list >>>>>>>>> AI(a)lists.wikimedia.org >>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>>>>> >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> AI mailing list >>>>>>>> AI(a)lists.wikimedia.org >>>>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> AI mailing list >>>>>>> AI(a)lists.wikimedia.org >>>>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>> AI mailing list >>>>> AI(a)lists.wikimedia.org >>>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>>> >>>> >>>> _______________________________________________ >>>> AI mailing list >>>> AI(a)lists.wikimedia.org >>>> https://lists.wikimedia.org/mailman/listinfo/ai >>>> >>>> >> >

1 0

The Revision Scoring weekly update
by Aaron Halfaker 13 Sep '16

13 Sep '16

Hey, This is the 21st weekly update from revision scoring team that we have sent to this mailing list. New development - We received a request to get moving on Spanish Wikibooks support, so we dug in: - We deployed a new Wiki labels campaign[1] - We fixed an issue in Wiki labels that prevented requests from *. wikibooks.org[2] - We trained a basic "revert" detection model that seems to be pretty effective[3] - We also generated a dataset of article quality scores for English Wikipedia[4]. You can download it here: [5] This week, we invested in some long term tasks. If you review our phabricator board, you'll see substantial progress in improving our damage detection models with hashing vectorization strategies[6, 7], implementing a more robust model testing strategy[8], and implementing some advance natural language processing strategies[9, 10]. Stay tuned for the completion of these activities in the coming weeks. 1. https://phabricator.wikimedia.org/T143962 -- Add uniqueness constraints to ores_classification 2. https://phabricator.wikimedia.org/T145406 -- Fix CORS for wikibooks 3. https://phabricator.wikimedia.org/T145428 -- Train/test reverted model for Spanish Wikibooks 4. https://phabricator.wikimedia.org/T135684 -- Generate recent article quality scores for English Wikipedia 5. https://datasets.wikimedia.org/public-datasets/enwiki/article_quality/wp10-… 6. https://phabricator.wikimedia.org/T128087 -- [Spike] Investigate HashingVectorizer 7. https://en.wikipedia.org/wiki/Feature_hashing 8. https://phabricator.wikimedia.org/T142953 -- Train on all data, Report test statistics on cross-validation 9. https://phabricator.wikimedia.org/T144636 -- Implement PCFG features 10. https://en.wikipedia.org/wiki/Stochastic_context-free_grammar Sincerely, Aaron from the Revision Scoring team

1 0

The Revision Scoring weekly update
by Aaron Halfaker 06 Sep '16

06 Sep '16

Hey, This is the 20th weekly update from revision scoring team that we have sent to this mailing list. New development: - We implemented the basic functionality for handling bag of words and other types of abstract feature vectors in `revscoring`. [1] This required some changes to some dependencies as well. [2] - We extended the user-group related features to include more of the dominant groups outside of English Wikipedia [3] and incremented the models that changed substantially [4] Documentation: - We extended the documentation at mw:Extension:ORES to make it easier for new developers to work with us. [5] Resourcing: - We discussed the teams resourcing needs (hardware, engineering, and community liaison support) with Wes Moran. [6] Maintenance and robustness: - We addressed a variety of issues around caching and how the ORES extension loads new data - ORES now returns headers that will disable secondary caching. [7] - Our maintenance scripts will circumvent caches that do not listen to no-cache headers. [8, 9] - We fixed an issue where the ORES review tool would duplicate items in Special:RecentChanges. [10] - We standardized the extraction pattern for the enwiktionary model so that it looks similar to other models. [11] 1. https://phabricator.wikimedia.org/T132580 -- Implement abstraction for Sparse Feature Vectors 2. https://phabricator.wikimedia.org/T144430 -- Update yamlconf so that import_path can handle deep attributes 3. https://phabricator.wikimedia.org/T143909 -- Extend user group features 4. https://phabricator.wikimedia.org/T144855 -- Increment ruwiki editquality models 5. https://phabricator.wikimedia.org/T144676 -- Improve technical documentation in Extension:ORES in mediawiki.ore 6. https://phabricator.wikimedia.org/T144517 -- ORES and Product: resourcing discussion 7. https://phabricator.wikimedia.org/T144193 -- Set max-age header to 0 seconds for ORES to quiet secondary caches 8. https://phabricator.wikimedia.org/T144196 -- Get model version needs to invalidate cache 9. https://phabricator.wikimedia.org/T144195 -- Check model version replaces every time it runs. 10. https://phabricator.wikimedia.org/T144233 -- Redundant results in ORES review tool 11. https://phabricator.wikimedia.org/T144605 -- Fix makefile entry for enwiktionary.rev_reverted.20k_2016.tsv Sincerely, Aaron from the Revision Scoring team

1 0

ORES article scores
by Pine W 04 Sep '16

04 Sep '16

Hi artificial intelligence people, Are there baseline ORES scores available for articles? If so, I'm thinking that it would be interesting to have the option to add a template to an article's talk page that shows its ORES score as well as its ORES percentile rank on a particular wiki. Pine

2 12

ORES review tool deployment status
by Aaron Halfaker 03 Sep '16

03 Sep '16

Hey folks, I recently received an email asking for more information about how to get the ORES review tool[1] deployed in more wikis (we currently support 8: wikidata, fawiki, enwiki, nlwiki, ptwiki, plwiki, trwiki, ruwiki). I figured that this summary should be shared more broadly, so I'm pasting it below. ---- This is the best guide that we have for users requesting support right now: https://meta.wikimedia.org/wiki/Objective_Revision_ Evaluation_Service/Get_support There's a lot that we need to do in response to requests for support. We currently have Wiki labels[2] edit quality campaigns running in http://labels.wmflabs.org/campaigns/arwiki/?campaigns=stats (2464/4977 labels) http://labels.wmflabs.org/campaigns/azwiki/?campaigns=stats (0/5000 labels) http://labels.wmflabs.org/campaigns/dewiki/?campaigns=stats (182/4177 labels) http://labels.wmflabs.org/campaigns/enwiki/?campaigns=stats (350/6333 labels, extension) http://labels.wmflabs.org/campaigns/eswiki/?campaigns=stats (1210/8434 labels) http://labels.wmflabs.org/campaigns/etwiki/?campaigns=stats (824/4678 labels) http://labels.wmflabs.org/campaigns/fawiki/?campaigns=stats (1781/3156 labels, extension) http://labels.wmflabs.org/campaigns/frwiki/?campaigns=stats (274/5000 labels) http://labels.wmflabs.org/campaigns/hewiki/?campaigns=stats (1069/5000 labels) http://labels.wmflabs.org/campaigns/huwiki/?campaigns=stats (518/5000 labels) http://labels.wmflabs.org/campaigns/idwiki/?campaigns=stats (50/2200 labels) http://labels.wmflabs.org/campaigns/itwiki/?campaigns=stats (591/5390 labels) http://labels.wmflabs.org/campaigns/jawiki/?campaigns=stats (356/9514 labels) http://labels.wmflabs.org/campaigns/nowiki/?campaigns=stats (1815/5000 labels) http://labels.wmflabs.org/campaigns/svwiki/?campaigns=stats (1657/5000 labels) http://labels.wmflabs.org/campaigns/ukwiki/?campaigns=stats (161/3318 labels) http://labels.wmflabs.org/campaigns/urwiki/?campaigns=stats (153/5000 labels) http://labels.wmflabs.org/campaigns/viwiki/?campaigns=stats (0/5000 labels) Note that we're developing a nice dashboard for this. See http://tools.wmflabs.org/dexbot/tools/wikilabels_stats.php Note that two campaigns are labeled "extension" because we already have support for those wikis, but we are running campaigns to extend the observations in our labeled datasets for higher fitness. In order to get these campaigns done, we need a local Wikipedian (or liasion) to call attention to the campaign and make sure that questions get answered and work continues. The wikis that already have support are those wikis where we found a strong local collaborator to help. - Wikidata - User:Ladsgroup - Ptwiki - User:He7d3r - Trwiki - User:WhiteCat - Enwiki - User:EpochFail - Fawiki - User:Ladsgroup - Ruwiki - User:Putnik - Nlwiki - User:Krinkle - Plwiki - User:Tar_Lócesilion As of right now, I'm the only person who is officially working on ORES full time. Amir (User:Ladsgroup) is funded to work on ORES 4 hours per week through WMDE. So, any time that someone asks something from us, we in turn, ask for support in order to be able to do it. The most critical support we could get for moving faster would be (1) community liaison support for identifying local collaborators and driving the current Wiki labels campaigns and (2) Engineering support to make the icky user-script[3] into a proper extension. 1. https://www.mediawiki.org/wiki/ORES_review_tool 2. https://meta.wikimedia.org/wiki/Wiki_labels 3. https://meta.wikimedia.org/wiki/Wiki_labels#Installation -Aaron

1 0

Training bias in machine learning
by Pine W 01 Sep '16

01 Sep '16

Perhaps of interest: http://www.seattletimes.com/business/microsoft/how-linkedins-search-engine-… "“Histories of discrimination can live on in digital platforms,” Kate Crawford, a Microsoft researcher, wrote <http://www.nytimes.com/2016/06/26/opinion/sunday/artificial-intelligences-w…> in The New York Times earlier this year. “And if they go unquestioned, they become part of the logic of everyday algorithmic systems.”" Pine

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

AI September 2016