Hi Joe,

The particular use case that I have in mind is that a reader, or potential contributor, wants to know the quality of a particular article. (Note that this is a slightly different use case from sorting many articles by their ORES scores, or comparing their ORES scores to the importance and/or manually assigned article quality grade.) ORES scores are likely to be more frequently updated than manual article quality grades, and I feel that this information would be useful to readers and/or potential contributors. Having the ORES score templates on talk pages, placed there by a bot and later refreshed weekly, would be adequate for the use case that I have in mind.

Note that https://upload.wikimedia.org/wikipedia/commons/6/6a/Wikimedia_Foundation_and_Reboot_New_Readers_Research_-_Nigeria_%26_India_Highlights_-_July_2016.pdf suggests that WMF Product "Add an indicator of quality icon (akin to the signal strength icon for mobile networks) on each article, based on research and analysis on article contributions and edits." I am thinking that ORES scores, and/or manual article quality grades, could serve this purpose. English Wikipedia already displays some article grades on the front pages of articles via the GA or FA icons, and my impression is that the New Readers Research suggests making some kind of quality indicator more prominent and on all Wikipedia wikis. This would involve work that's beyond the scope of what I have in mind, but I think that it's worth noting for potential future development. Abbey may wish to comment on this.

Pine

On Fri, Sep 16, 2016 at 2:06 PM, Joe Matazzoni <jmatazzoni@wikimedia.org> wrote:
The project we’re working on with ORES (Edit Review Improvements ) is concerned only with the Damaging and Good Faith scores at this time. So I don’t know too much about the ORES article scores. I believe they have a lot of potential, but I’m a little unclear on the general goal of this proposal.

My sense of the ORES article scores is that they’re most useful for people who are not (yet) looking at the article. E.g., the scores might be used to review AfC backlogs and surface the pages that are ready for re-review. Or they might be used to help reviewers engaged in finding and promoting Featured Articles. They might conceivably have a place in search results some day, as additional data for readers. 

My guess is that these scores are based at this time on formal characteristics of the article — its length, its conformance with formatting standards, whether it includes photos or multimedia, whether it has references, etc. These formal characteristics would for the most part, I’d think,  be obvious to someone already on the article page.

But ORES does not know if a subject is notable, for example.  Nor do t believe it would know if  an article were factually correct, among other things. 

So, again, what is your goal? Is there a Phabricator task? Good luck!

Joe



_____________________

Joe Matazzoni 
Product Manager, Collaboration
Wikimedia Foundation, San Francisco
mobile 202.744.7910
jmatazzoni@wikimedia.org

"Imagine a world in which every single human being can freely share in the sum of all knowledge." 




On Sep 15, 2016, at 8:57 PM, Pine W <wiki.pine@gmail.com> wrote:

Joe, any comment about this?

Toby and Amir, would it make sense to integrate ORES scores into https://meta.wikimedia.org/wiki/Community_Tech/PageAssessments, or have some other kind of relation between the ability to pull ORES scores from the MW API and the PageAssessments API? Or does it make sense to have them be separate but working in parallel? I'm an API novice, but the functionality of PageAssessments reminds me of what we're discussing here, which is why I bring it up.

Thanks,

Pine

On Wed, Aug 31, 2016 at 2:48 PM, Toby Negrin <tnegrin@wikimedia.org> wrote:
Hi Pine -- thanks for this. I'm looping in Joe M. from the Collaboration team who can speak to his team's thoughts on ORES scores.

It seems likely that my team will devote some resources to making ORES (+ the page view API) accessible via the MW API next quarter which will help integration efforts. Our goals go up on Friday and I just got out of a meeting where we agreed to do this and it's unlikely this will change in 2 days.

As far as New Readers though, I suspect that it's a bit too abstract as of yet. I think we should start with Old Readers :)

-Toby

On Wed, Aug 31, 2016 at 2:37 PM, Pine W <wiki.pine@gmail.com> wrote:
Just to note that https://upload.wikimedia.org/wikipedia/commons/6/6a/Wikimedia_Foundation_and_Reboot_New_Readers_Research_-_Nigeria_%26_India_Highlights_-_July_2016.pdf suggests that WMF Product "Add an indicator of quality icon (akin to the signal strength icon for mobile networks) on each article, based on research and analysis on article contributions and edits." This is similar to what I have in mind for talk pages with ORES scores. Adding indicators to the front page of articles would likely require an RfC which may be more involved than an RfB to add the score to talk pages. Still, I can see how having an ORES score, or the Wikipedia Quality score (at least on ENWP) indicated by an icon at the top of article pages would be useful, especially if readers can intuitively understand what the indicator is telling them without needing to look at documentation.

I think it might be best to start with ORES scores on talk pages. That fills the need that I have in mind. If someone wants to extend this to having article quality indicators on the front page of articles, I'd be happy to collaborate with them.

Pine

On Tue, Aug 16, 2016 at 7:32 AM, Pine W <wiki.pine@gmail.com> wrote:

Thanks. I will add this to my agenda for later this year.

Pine


On Aug 16, 2016 01:48, "Amir Ladsgroup" <ladsgroup@gmail.com> wrote:
Oh, Sorry I missed the question. Yes, it's fairly easy :)

Best

On Tue, Aug 16, 2016 at 12:15 PM Pine W <wiki.pine@gmail.com> wrote:

Pinging again to repeat my question from my previous email.

Thanks,
Pine


On Aug 9, 2016 16:04, "Pine W" <wiki.pine@gmail.com> wrote:
I'd like to work on this if I had the time to learn and to do it. Maybe I will in early September or sometime later this year. I've put it on my to do list. This would be my first bot; do you think that it's a good project for a newbie bot builder?

Pine

On Sat, Aug 6, 2016 at 11:16 AM, Amir Ladsgroup <ladsgroup@gmail.com> wrote:
Building that is easy, you just need someone to add these functionalities to the templates and a bot to run it for the first time.

Any volunteers?

Best

On Mon, Aug 1, 2016 at 9:37 PM Pine W <wiki.pine@gmail.com> wrote:
Hi Amir,

I think that I'm following better now.

The extension option doesn't sound workable for what I have in mind. I'd like the score to be visible without needing to be logged in, which I believe means that the feature can't be conditional on having access to beta features.

The solution to click on a talk page link for an updated score sounds more workable. It would be best if the score refreshed automatically every time someone viewed the talk page for the purpose that I have in mind, but I don't know what kind of performance penalty and computing resources that would require. If either of those are nontrivial, then having a template on the talk page that says something like "Score last refreshed on TIME DATE. There have been X revisions since that time. Click here to update the score" would be good for the use case that I have in mind.

Pine

On Mon, Aug 1, 2016 at 1:48 AM, Amir Ladsgroup <ladsgroup@gmail.com> wrote:
Hey,


On Sun, Jul 31, 2016 at 8:16 AM Pine W <wiki.pine@gmail.com> wrote:
Hi Amir,

Thanks for waiting. As I understand it, we have these options for making the ORES scores visible on article talk pages:

1. A gadget
2. An addition to the ORES extension
3. A bot

Is that correct?
Yes 

WP:WPP refers to WikiProject Philosophy; did you man something else or am I misunderstanding the abbreviation?

My bad, I meant something like this: https://en.wikipedia.org/wiki/Template:WikiProject_Physics 
I'm thinking that a gadget might be a workable solution, but my preference would be to have the score displayed on all article talk pages automatically for everyone to see including readers who are not logged in. A bot that posted scores to talk pages and updated the scores on a weekly basis would be good for that purpose.

Both are possible solutions but as for bots. Updating the score for all articles of a wikiproject (e.g. Wikiproject medicine has 35K articles) on weekly basis is not efficient. That bot needs to do 35K edits every week and it might make some people mad. We can have an initial release and update it on demand (like adding a link to the score in the template. If someone wants that score gets updated. Just clicks on the link and the bot update it)
I'm unclear on how an addition to the ORES extension would work; can you explain further what that option would involve?
Adding it to the ores extension will make things much easier. You just need to enable it in your beta features and then in the talk page or action=info you will see the score. Implementing it is difficult due to storage issues for the database and we might need to get operations approval before continuing. 

Best

Thanks,

Pine

On Sun, Jul 24, 2016 at 11:29 PM, Pine W <wiki.pine@gmail.com> wrote:
Hi Amir,

Thanks for getting back to me. My mind is a little full at the moment, and I have some deadlines this week, so it may take me a little while to get back to you. But I'm flagging this conversation for follow up, hopefully by the end of this week or early next week.

Thanks for your patience,
Pine

On Fri, Jul 22, 2016 at 9:59 PM, Amir Ladsgroup <ladsgroup@gmail.com> wrote:
Hey,
ORES has "article quality" models for English, Russian and French Wikipedia. There is a very short introduction in ORES description page.
But if I want to explain it my way. It'll give you a prediction of class of the article (A, B, C, FA, GA, stub, start) for a given revision. For example for this revision  ORES predicts that it's GA (which it is). 

Your use case about ORES prediction is valid the second part is a little bit complicated though. For the first part, we have either option of a gadget or adding it to the ORES extension. If it's very important to let everyone see the scores, we can write a bot to update reports weekly in English Wikipedia for articles in a certain Wikiproject and we use these data in the WPP templates. Which one do you think suits English Wikipedia better?

Best


On Sat, Jul 23, 2016 at 2:57 AM Pine W <wiki.pine@gmail.com> wrote:
Hi artificial intelligence people,

Are there baseline ORES scores available for articles? If so, I'm thinking that it would be interesting to have the option to add a template to an article's talk page that shows its ORES score as well as its ORES percentile rank on a particular wiki.

Pine
_______________________________________________
AI mailing list
AI@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/ai

_______________________________________________
AI mailing list
AI@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/ai



_______________________________________________
AI mailing list
AI@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/ai

_______________________________________________
AI mailing list
AI@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/ai


_______________________________________________
AI mailing list
AI@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/ai

_______________________________________________
AI mailing list
AI@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/ai


_______________________________________________
AI mailing list
AI@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/ai

_______________________________________________
AI mailing list
AI@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/ai