Hi folks,
there are many projects which have an interest in generating and querying metadata for specific revisions:
- community efforts to annotate quality of specific articles - researchers analyzing revision contents (e.g. to derive quality heuristics, perform citation analysis, etc.) - application developers wanting to display/use such metadata
We've gotten inquiries e.g. from WikiProjects and researchers over the years.
I'm wondering if a lightweight service that satisfies the following requirements might be a good idea:
- community-created schemas (similar to the EventLogging schemas on meta) - basic per-user authentication/authorization - basic namespacing (e.g. "WikiProject Medicine:Quality" refers to a specific schema + specific permissions)
If such a service existed, community members, researchers and occasionally WMF itself could create their own tools/gadgets that use this service, perhaps with a lightweight global approval process.
If this seems like a good idea, I'd be curious about implementation strategies -- are we blocked on something like SOA Auth [1] to implement this as a standalone service? My sense is that you'd want to pull this out of MediaWiki for maximum flexibility and simplicity.
Thanks, Erik
[1] https://www.mediawiki.org/wiki/Requests_for_comment/SOA_Authentication
I wonder if this might be useful for per-page glossaries? https://www.mediawiki.org/wiki/Requests_for_comment/Scoped_language_converte... --scott
Erik,
On 11/05/2014 10:07 AM, Erik Moeller wrote:
I'm wondering if a lightweight service that satisfies the following requirements might be a good idea:
- community-created schemas (similar to the EventLogging schemas on meta)
- basic per-user authentication/authorization
- basic namespacing (e.g. "WikiProject Medicine:Quality" refers to a
specific schema + specific permissions)
the need for storing different formats and metadata per revision was actually one of the motivations for creating RESTBase [1]. Currently it is set up to store html, wikitext, data-parsoid and data-mw per revision, with each property being stored in its own bucket behind the scenes. It is possible to add new revisioned buckets for new types of content with a simple PUT, and the plan is to have separate ACLs per bucket.
What are the indexing requirements for this metadata? If fast access by specific properties is needed, then using tables would make more sense, as we'll then be able to leverage secondary indexing. Tables have the same properties as buckets, and can also be created with a PUT of the schema. Query results are returned as JSON.
A limitation for queries in RESTBase is that they are limited to indexes defined in the schema. If ad-hoc queries on arbitrary combinations of attributes are needed, then ElasticSearch would be more suitable.
If such a service existed, community members, researchers and occasionally WMF itself could create their own tools/gadgets that use this service, perhaps with a lightweight global approval process.
If this seems like a good idea, I'd be curious about implementation strategies -- are we blocked on something like SOA Auth [1] to implement this as a standalone service? My sense is that you'd want to pull this out of MediaWiki for maximum flexibility and simplicity.
It might be possible to improvise a bit, but we'll need basic SOA auth fairly soon for other use cases too. I'm optimistic that we can start small though, especially if this doesn't need to tie into browser-based SUL straight away.
Gabriel
On Wed, Nov 5, 2014 at 11:21 AM, Gabriel Wicke gwicke@wikimedia.org wrote:
What are the indexing requirements for this metadata? If fast access by specific properties is needed
Most typically, I'm guessing you'd do stuff on a per-revision basis to show quality indicators and such on page histories or article pages via opt-in gadgets. Querying the entire corpus for articles with certain characteristics would be valuable though, especially for applications like offline exports.
I just saw https://meta.wikimedia.org/wiki/Grants:IEG/Revision_scoring_as_a_service and wasn't even aware of that when I wrote the email -- there's definitely a lot of interest in a generic solution for this problem.
Erik
We seem to really gravitate towards complexity on these things. How can we make them simple, addressing a very specific need. We can complicate later.
Here is a scenario (which we should start with, not architecture)
1. As an editor I'd like to flag a revision as reviewed/verified by me from the revision screen or list. 2. As an editor I want to see which revisions were verified/had second opinion by other editors.
*So instead of a long spec that attempts to solve for a ton of cases, let's start thinking about solving simple, direct pain-points, iteratively.*
Can we do that?
L
On Fri, Nov 7, 2014 at 8:40 PM, Erik Moeller erik@wikimedia.org wrote:
On Wed, Nov 5, 2014 at 11:21 AM, Gabriel Wicke gwicke@wikimedia.org wrote:
What are the indexing requirements for this metadata? If fast access by specific properties is needed
Most typically, I'm guessing you'd do stuff on a per-revision basis to show quality indicators and such on page histories or article pages via opt-in gadgets. Querying the entire corpus for articles with certain characteristics would be valuable though, especially for applications like offline exports.
I just saw https://meta.wikimedia.org/wiki/Grants:IEG/Revision_scoring_as_a_service and wasn't even aware of that when I wrote the email -- there's definitely a lot of interest in a generic solution for this problem.
Erik
-- Erik Möller VP of Product & Strategy, Wikimedia Foundation
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I suspect it isn't done because it isn't a very good way to modify a complex embedded base of software, Lila. Generally, when modifying a complex embedded base, one designs first, iterates implementation and internal testing, and then releases a relatively complete piece of functionality.
KWW
Lila Tretikov schreef op 2014/11/08 12:37:
We seem to really gravitate towards complexity on these things. How can we make them simple, addressing a very specific need. We can complicate later.
Here is a scenario (which we should start with, not architecture)
1. As an editor I'd like to flag a revision as reviewed/verified by me from the revision screen or list. 2. As an editor I want to see which revisions were verified/had second opinion by other editors.
*So instead of a long spec that attempts to solve for a ton of cases, let's start thinking about solving simple, direct pain-points, iteratively.*
Can we do that?
L
On Fri, Nov 7, 2014 at 8:40 PM, Erik Moeller erik@wikimedia.org wrote:
On Wed, Nov 5, 2014 at 11:21 AM, Gabriel Wicke gwicke@wikimedia.org wrote:
What are the indexing requirements for this metadata? If fast access by specific properties is needed
Most typically, I'm guessing you'd do stuff on a per-revision basis to show quality indicators and such on page histories or article pages via opt-in gadgets. Querying the entire corpus for articles with certain characteristics would be valuable though, especially for applications like offline exports.
I just saw https://meta.wikimedia.org/wiki/Grants:IEG/Revision_scoring_as_a_service and wasn't even aware of that when I wrote the email -- there's definitely a lot of interest in a generic solution for this problem.
Erik
-- Erik Möller VP of Product & Strategy, Wikimedia Foundation
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hey Erik,
I'm glad to see that we're imagining similar things. :) This project has been on my to-do list for years.
I don't think that building a well-designed service and starting in labs are conflicting options. Regardless this project is marching forward in the next couple of months. I don't think we're breaking any new ground conceptually. Really, we're just building a feature extractor, gathering hand-coded revisions and training a standard classifier. Effective strategies for doing all of these things are well described in the literature and demonstrated in STiki.
Lila, your proposal seems to be orthogonal to the project I proposed in the grant. Petr, the developer/maintainer of the current version of Huggle, has been calling for such a service as you describe for years. See http://www.gossamer-threads.com/lists/wiki/wikitech/392727?do=post_view_thre...
-Aaron
On Sat, Nov 8, 2014 at 11:52 PM, Kevin Wayne Williams < kwwilliams@kwwilliams.com> wrote:
I suspect it isn't done because it isn't a very good way to modify a complex embedded base of software, Lila. Generally, when modifying a complex embedded base, one designs first, iterates implementation and internal testing, and then releases a relatively complete piece of functionality.
KWW
Lila Tretikov schreef op 2014/11/08 12:37:
We seem to really gravitate towards complexity on these things. How can we
make them simple, addressing a very specific need. We can complicate later.
Here is a scenario (which we should start with, not architecture)
1. As an editor I'd like to flag a revision as reviewed/verified by me from the revision screen or list. 2. As an editor I want to see which revisions were verified/had second opinion by other editors.
*So instead of a long spec that attempts to solve for a ton of cases, let's start thinking about solving simple, direct pain-points, iteratively.*
Can we do that?
L
On Fri, Nov 7, 2014 at 8:40 PM, Erik Moeller erik@wikimedia.org wrote:
On Wed, Nov 5, 2014 at 11:21 AM, Gabriel Wicke gwicke@wikimedia.org
wrote:
What are the indexing requirements for this metadata? If fast access by
specific properties is needed
Most typically, I'm guessing you'd do stuff on a per-revision basis to show quality indicators and such on page histories or article pages via opt-in gadgets. Querying the entire corpus for articles with certain characteristics would be valuable though, especially for applications like offline exports.
I just saw https://meta.wikimedia.org/wiki/Grants:IEG/Revision_scoring_as_a_service and wasn't even aware of that when I wrote the email -- there's definitely a lot of interest in a generic solution for this problem.
Erik
-- Erik Möller VP of Product & Strategy, Wikimedia Foundation
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hey,
I suspect it isn't done because it isn't a very good way to modify a
complex embedded base of software, Lila. Generally, when modifying a complex embedded base, one designs first, iterates implementation and internal testing, and then releases a relatively complete piece of functionality.
As far as I understand this is about adding a new service, not modifying complex existing behaviour. Is that wrong?
Also, while I can't speak for Lila, it does seem to me she is suggesting to go with a simple solution that tackles a specific need well, as opposed to releasing something incomplete. The idea to build small dedicated units rather than monoliths is not something new or controversial in the world of software design. Even though it might be in certain insular communities.
See also: https://en.wikipedia.org/wiki/Unix_philosophy
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate Evil software architect at Wikimedia Germany ~=[,,_,,]:3
Jeroen De Dauw schreef op 2014/11/09 0:29:
Hey,
I suspect it isn't done because it isn't a very good way to modify a
complex embedded base of software, Lila. Generally, when modifying a complex embedded base, one designs first, iterates implementation and internal testing, and then releases a relatively complete piece of functionality.
As far as I understand this is about adding a new service, not modifying complex existing behaviour. Is that wrong?
Also, while I can't speak for Lila, it does seem to me she is suggesting to go with a simple solution that tackles a specific need well, as opposed to releasing something incomplete. The idea to build small dedicated units rather than monoliths is not something new or controversial in the world of software design. Even though it might be in certain insular communities.
Her comment included
1. As an editor I'd like to flag a revision as reviewed/verified by me from the revision screen or list. 2. As an editor I want to see which revisions were verified/had second opinion by other editors.
That's not a "service" by any definition I'm aware of, that's a user interaction, one that seems to presume the existence of the Wikimedia base.
There's no doubt that there's a lot of leeway in terms of how much one implements in each release of software, but one should always have a reasonably clear image of the final target, know where the piece being implemented now fits into the final result, and take steps to ensure that each release is sufficiently complete to be useful. There have been some successful products built using the "code first, design later" method, but that doesn't mean it should be encouraged. Even SCRUM encourages you to have some idea of what the final result will be, even if each phase is scoped opportunistically.
KWW
Hey,
1. As an editor I'd like to flag a revision as reviewed/verified by me
from the revision screen or list.
- As an editor I want to see which revisions were verified/had second opinion by other editors.
That's not a "service" by any definition I'm aware of, that's a user interaction, one that seems to presume the existence of the Wikimedia base.
You are right. I was thinking about what has been described in the first post of this thread.
There have been some successful products built using the "code first, design later" method, but that doesn't mean it should be encouraged.
Fully agree. Writing code and designing are things that go together. If you write code without thinking about its design, you'll end up with bad code. If you design without ever writing code, your designs will likely not hold up well in the real world.
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate Evil software architect at Wikimedia Germany ~=[,,_,,]:3
Lila Tretikov wrote:
We seem to really gravitate towards complexity on these things. How can we make them simple, addressing a very specific need. We can complicate later.
Our wikis range in size from a few hundred revisions to hundreds of millions of revisions (and growing!). Building a scalable system is hard work and the Wikimedia Foundation has been pretty aggressive in focusing primarily on large wikis (Commons and the English Wikipedia), so perhaps most technical discussions tend to shift to the higher gears as a result.
- As an editor I'd like to flag a revision as reviewed/verified by me
from the revision screen or list.
- As an editor I want to see which revisions were verified/had second
opinion by other editors.
Okay, so... we already have this feature built in to MediaWiki core (it's documented at https://meta.wikimedia.org/wiki/Help:Patrolled_edit) and we did the whole "try to flag revisions as reviewed" bit in 2008 and 2009 with a MediaWiki extension called FlaggedRevisions (re-branded as "Pending Changes" for added confusion).
I'm not sure what you're bringing to the table here. We've already made substantial investments in developing revision review technology that didn't work well as it was seemingly incompatible with the wiki model.
MZMcBride
Federico Leva (Nemo) wrote:
FlaggedRevisions has never been rebranded.
If you say so.
https://www.mediawiki.org/wiki/Pending_Changes_enwiki_trial says "Deploy FlaggedRevs extension to en.wikipedia.org" while https://blog.wikimedia.org/?p=3819 says "As mentioned in this post in January, the English Wikipedia will be trying out the Flagged Revisions extension, using a configuration we’re calling Pending Changes."
I seem to specifically remember some Wikimedia Foundation middle-management coming along and insisting that FlaggedRevs be called Pending Changes in order to be deployed to the English Wikipedia, but it's possible I'm simply misremembering. Posts such as https://blog.wikimedia.org/?p=3843 seem to confirm that the terms were used interchangeably.
MZMcBride
Yes, a failed piece of rotting [configuration] code on en.wiki is called "Pending changes"; this doesn't mean that the extension, used by over 200 wikis, has been affected in any way.
Nemo
As someone who patrols recent changes to our 33,000 (and rising) health sciences-related articles with a diminishing number of colleagues, one thing developers could do to help us keep that content safe would be this:
Allow us to tag the log entry of normal revisions (not pending changes/flagged revisions - no medical articles presently have flagged revisions, and none are likely to in the near future) as having been reviewed for policy/guideline compliance by a trusted editor.
There are maybe a dozen regular/semi-regular med patrollers (down from about twice that number three years ago), and I'm very conscious that we aren't keeping up. If I see a revision has been reviewed by one of that dozen whom I trust, I'll (not always, but often) skip checking that revision and move on to the next unreviewed revision.
This will
a) save me and the others a lot of time, allowing us to cover much more ground and b) give us a handle on how thoroughly we're vetting changes to this sensitive content.
Ideally, each of us that reviews a revision should be able to tag its log entry - so we can see the depth of review each revision has undergone.
The board of Wiki Project Med Foundation are discussing this at the moment, and we see it as a very effective step toward safeguarding and improving our medical offering. If you could do this for us, it would be very much appreciated.
Anthony Cole http://en.wikipedia.org/wiki/User_talk:Anthonyhcole
On Mon, Nov 10, 2014 at 11:25 PM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Yes, a failed piece of rotting [configuration] code on en.wiki is called "Pending changes"; this doesn't mean that the extension, used by over 200 wikis, has been affected in any way.
Nemo
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
W dniu środa, 12 listopada 2014 Anthony Cole ahcoleecu@gmail.com napisał(a):
Allow us to tag the log entry of normal revisions (not pending changes/flagged revisions - no medical articles presently have flagged revisions, and none are likely to in the near future) as having been reviewed for policy/guideline compliance by a trusted editor.
This is one of the things the FlaggedRevs extension (the same one that powers the "pending changes" system on the English Wikipedia) allows you to do. It can be configured to provide arbitrary flags (not just binary "okay"/"not okay"), and it can be configured to display the latest version of the article (rather than the "flagged" one) to visitors by default, and it can be configred to work on all articles on a wiki.
Thanks Bartosz. Just to clarify:
If we apply FlaggedRevs to all medical articles (articles that have the WP:MED template on their talk page), configured to display the latest article version, can we create a permission (say, Medicine Reviewer) that allows one to tag the revision log entry with a comment? Would it interfere in any way with the normal practice of other editors who don't have that permission?
Anthony Cole http://en.wikipedia.org/wiki/User_talk:Anthonyhcole
On Wed, Nov 12, 2014 at 3:05 AM, Bartosz Dziewoński matma.rex@gmail.com wrote:
W dniu środa, 12 listopada 2014 Anthony Cole ahcoleecu@gmail.com napisał(a):
Allow us to tag the log entry of normal revisions (not pending changes/flagged revisions - no medical articles presently have flagged revisions, and none are likely to in the near future) as having been reviewed for policy/guideline compliance by a trusted editor.
This is one of the things the FlaggedRevs extension (the same one that powers the "pending changes" system on the English Wikipedia) allows you to do. It can be configured to provide arbitrary flags (not just binary "okay"/"not okay"), and it can be configured to display the latest version of the article (rather than the "flagged" one) to visitors by default, and it can be configred to work on all articles on a wiki.
-- -- Matma Rex _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Note that I am not an expert, really – I hail from the Polish Wikipedia which has FlaggedRevs enabled for all articles (using a pretty simple configuration), and I played with it a bit on a private wiki.
On Thu, 13 Nov 2014 02:38:00 +0100, Anthony Cole ahcoleecu@gmail.com wrote:
If we apply FlaggedRevs to all medical articles (articles that have the WP:MED template on their talk page)
I'm not sure if this is possible, I think you currently have to make it possible to flag revisions of all articles on the wiki. This could possibly be developed, or could just be "enforced" using common sense, since these are trusted editors we're talking about anyway.
configured to display the latest article version, can we create a permission (say, Medicine Reviewer) that allows one to tag the revision log entry with a comment?
As far as I know, yes.
Would it interfere in any way with the normal practice of other editors who don't have that permission?
As far as I know, no. The information on whether a revision was flagged or not might be shown in the UI somewhere, but it shouldn't cause any editing problems, or any fancy buttons to appear for people without the permission to review.
wikitech-l@lists.wikimedia.org