From: Daniel P. B. Smith [mailto:dpbsmith@verizon.net]
Back in June, I complained that a little particle of misinformation from Wikipedia had gotten lodged in my brain, and might potentially have affected my car-purchasing decision. Specifically, I was referring to an article that characterized the Toyota Echo, as a "flop" in the U.S. whose sales had tanked in 2004 and was due to be discontinued--despite continuing success in many other countries, including Canada.
When Wikipedia fails, this is the typical way it does fail. Someone makes insufficient effort to balance the article.
Now that summer is over, I will renew my Quarterly Qall for Quality (oops, typo! ;-) at Wikipedia.
We need a kind of rating scheme or quality assurance system or certification mechanism. It's not a problem of designing the software for this. Tim and Erik and Magnus and Brion and all the rest are superbly capable. But *they* are not going to take the lead in this. *We* must decide that we want some means of assuring readers that they are getting reliable information.
Once again, I suggest that we let users "review" a given "version" of an article and (using a "new" software feature) "mark" that version as:
* "patrolled" (as in Recent Changes "simple vandalism" patrol) * "accurate" (i.e. "I am personally convinced that Everything this article says is true and correct." * "balanced" (i.e., nothing has been left out or downplayed)
For problems: * "graffiti" (or "vandalism" = someone has messed up this version, but I don't have the time, inclination or ability to undo the damage) * "inaccurate" (contains mistakes, which we *hope* they'll mention on the talk page) * "bias" (tells one side of a story, especially in a raging controversy)
Now where we take it from here is really up for grabs. Some people won't even care about these tags. We can set the default to ignore quality tags unless you "opt in".
RC patrollers might like to know WHO has reviewed a version. If Mav says he's checked the diff for "simple vandalism", I wouldn't give it a second thought. He's the champ. He used to check EVERY change (!) when traffic was slow enough. Now that there are often 100s of edits per minute, this work needs to be split up. I would be happy to put in an hour, from time to time, if only I knew "who else" had certified a certain article version as "patrolled". I'd ignore known troublemakers, for example. Double check newbies, and not even bother reviewing the work of people I come to trust.
Knowing that a particular VERSION of an article had been certified as "balanced" would help Administrators if an edit war flares up. If they need to protect the article, they could go back to the last version which had been marked "balanced" by someone they trust. To be fair, they would almost certainly have to pick a version certified by someone other than themselves - unless they weren't involved in the dispute.
For the looming print version (or CD / DVD version), we could automatically choose article versions which have a suitable combination of Approval Tags and Problem tags. (My own preference would be "patrolled" and no "graffiti".) A library might insist on "accurate" and "balanced" with no "inaccurate" or "bias".
We can let people use the tags as a filtering system. When browsing or looking up information, you might want to see:
* the most recent version which HAS the tags you like;
Or
* the most recent version which DOES NOT HAVE any of the tags you hate
Or
* the most recent version which has all the tags you like and none of the tags you hate.
But when you went to edit, you'd see the latest version same as before. Perhaps you'd get a notice saying "This is the latest version. You were looking at yesterday's / last week's version."
What do you all think?
On 06/09/05, Poor, Edmund W Edmund.W.Poor@abc.com wrote:
RC patrollers might like to know WHO has reviewed a version. If Mav says he's checked the diff for "simple vandalism", I wouldn't give it a second thought. He's the champ. He used to check EVERY change (!) when traffic was slow enough. Now that there are often 100s of edits per minute, this work needs to be split up. I would be happy to put in an hour, from time to time, if only I knew "who else" had certified a certain article version as "patrolled". I'd ignore known troublemakers, for example. Double check newbies, and not even bother reviewing the work of people I come to trust.
I haven't used it, but doesn't CDVF allow this functionality to some degree, with whitelisting &c?
For the looming print version (or CD / DVD version), we could automatically choose article versions which have a suitable combination of Approval Tags and Problem tags. (My own preference would be "patrolled" and no "graffiti".) A library might insist on "accurate" and "balanced" with no "inaccurate" or "bias".
Note that, thanks to transclusion, "no graffiti" does not mean it doesn't have giant images of penises stuck everwhere across it...
On 9/6/05, Poor, Edmund W Edmund.W.Poor@abc.com wrote:
From: Daniel P. B. Smith [mailto:dpbsmith@verizon.net]
Back in June, I complained that a little particle of misinformation from Wikipedia had gotten lodged in my brain, and might potentially have affected my car-purchasing decision. Specifically, I was referring to an article that characterized the Toyota Echo, as a "flop" in the U.S. whose sales had tanked in 2004 and was due to be discontinued--despite continuing success in many other countries, including Canada.
When Wikipedia fails, this is the typical way it does fail. Someone makes insufficient effort to balance the article.
In this case, there was no balance to add. Several sources were claiming the Echo's demise in the US market, and none were claiming it would continue.
Now that summer is over, I will renew my Quarterly Qall for Quality
(oops, typo! ;-) at Wikipedia.
We need a kind of rating scheme or quality assurance system or certification mechanism. It's not a problem of designing the software for this. Tim and Erik and Magnus and Brion and all the rest are superbly capable. But *they* are not going to take the lead in this. *We* must decide that we want some means of assuring readers that they are getting reliable information.
Once again, I suggest that we let users "review" a given "version" of an article and (using a "new" software feature) "mark" that version as:
- "patrolled" (as in Recent Changes "simple vandalism" patrol)
- "accurate" (i.e. "I am personally convinced that Everything
this article says is true and correct."
- "balanced" (i.e., nothing has been left out or downplayed)
For problems:
- "graffiti" (or "vandalism" = someone has messed up this version,
but I don't have the time, inclination or ability to undo the damage)
- "inaccurate" (contains mistakes, which we *hope* they'll mention on
the talk page)
- "bias" (tells one side of a story, especially in a raging controversy)
In this case, tagging an article would cause more problems than it would solve. When information changes, we don't want the "certified valid" version to be based on out of date information.
Also, certification is simply inserting a POV into the article's metadata. It is a reader's POV, but nonetheless, that's what it is.
Now where we take it from here is really up for grabs. Some people won't
even care about these tags. We can set the default to ignore quality tags unless you "opt in".
RC patrollers might like to know WHO has reviewed a version. If Mav says he's checked the diff for "simple vandalism", I wouldn't give it a second thought. He's the champ. He used to check EVERY change (!) when traffic was slow enough. Now that there are often 100s of edits per minute, this work needs to be split up. I would be happy to put in an hour, from time to time, if only I knew "who else" had certified a certain article version as "patrolled". I'd ignore known troublemakers, for example. Double check newbies, and not even bother reviewing the work of people I come to trust.
Knowing that a particular VERSION of an article had been certified as "balanced" would help Administrators if an edit war flares up. If they need to protect the article, they could go back to the last version which had been marked "balanced" by someone they trust. To be fair, they would almost certainly have to pick a version certified by someone other than themselves - unless they weren't involved in the dispute.
I don't see the value in article certification, as it's just another form of voting. With as fast as articles change in Wikipedia, I could spend a whole day every week "recertifying" articles on my watch list. On busy articles, most of them would get edited again before any significant number of people cast their votes on them, so we'd have a dozen or two versions in the article history, each with one vote. I don't see value in that.
Further, would such article voting appear in recent changes and watchlists? I watch almost everything I've voted on, so I can explain my vote or discuss the votes of others. I don't think I'd expect this to be any different.
I would rather have more personalized watch lists. When I watch an article, I want to be able to review the latest edits, then "check off" that article in my watch list to say "I'm OK with those edits, take this article off my watch list until the article changes again" or not "check off" the article so that I may review it more carefully later. If items were removed from my watchlist as I checked them, I could probably watch 10 times as many items or more.
For the looming print version (or CD / DVD version), we could
automatically choose article versions which have a suitable combination of Approval Tags and Problem tags. (My own preference would be "patrolled" and no "graffiti".) A library might insist on "accurate" and "balanced" with no "inaccurate" or "bias".
We can let people use the tags as a filtering system. When browsing or looking up information, you might want to see:
- the most recent version which HAS the tags you like;
Or
- the most recent version which DOES NOT HAVE any of the tags you hate
Or
- the most recent version which has all the tags you like and none of
the tags you hate.
But when you went to edit, you'd see the latest version same as before. Perhaps you'd get a notice saying "This is the latest version. You were looking at yesterday's / last week's version."
What do you all think?
The short version of my opinion: I don't think this is capable of keeping up with Wikipedia's changes, not because of the software, but because of limited editor time.
I think if the watchlists are made more efficient, we'll get a far better return on the developers' time investment.
Poor, Edmund W wrote:
What do you all think?
I think we should stick to Jimbos plan and turn the validation feature loose and see what happens; then decide base on the data we gather.
It was working OK before Wikimania, and still should...
I know, Brion is busy with the XML export/import and a zillion other things, but if you find a few minutes, maybe you could give it a quick check and turn it on? If it drags the servers down, it can always be turned off again...
Magnus
[people on wikien-l wanting the article rating feature switched on]
Magnus Manske (magnus.manske@web.de) [050907 00:40]:
I know, Brion is busy with the XML export/import and a zillion other things, but if you find a few minutes, maybe you could give it a quick check and turn it on? If it drags the servers down, it can always be turned off again...
I understand he didn't like it pulling in all revisions (that being the aspect which would bring the DB to a grinding halt). Perhaps pull in just the last ten, and let people page back through earlier revs ten at a time? And if rating articles is HUGELY popular, change that to five revs at a time ;-)
http://mail.wikimedia.org/pipermail/wikitech-l/2005-June/030325.html
- the second and third problems there can be worked around by requiring steward or wikisysop-level access to mess with stuff; changing rating categories would presumably be a fantastically rare action.
- d.
On 9/6/05, Poor, Edmund W Edmund.W.Poor@abc.com wrote:
- "accurate" (i.e. "I am personally convinced that Everything
this article says is true and correct."
I don't want to start a thread about epistemology, but personally there are NO articles where I am personally convinced that *everything* the article says is true and correct (and by "true" and "correct", I don't even mean in an ontological sense -- just the standard "this is what is generally thought to be true" subjective sense). Even on the topics I know very well -- even the articles I myself have written -- I would never consider my faith in an article higher than "This article contains nothing which I know to be blatantly incorrect, though it's always possible that various errors have snuck in one way or another." Typos, thinkos, misinformed sources, etc. all can lead to accidental and unintentional errors, much less ones deliberately introduced for whatever reason. But there's little on my watchlist which I think contains blatantly false information, which is something, at least... just my two cents.
FF
Ed Poor wrote
*We* must decide that we want some means of assuring readers that they
are getting reliable information.
Simple - just protect decent, informative versions of pages. True, this turns a wiki into a standard static web site; but hey, people will be mightily assured.
I'm sort of amazed that the category system has't already been adapted to give some sort of 'marks out of 10' to articles. If there were real community pressure to rate articles, I think that might have happened already. What I see is only the skeleton of such as system: stub tags, featured-article status, and the reprehensible use of POV tags as comment rather than trying to sort out disputes.
Trying to think laterally for the moment. If articles were rated on a scale of 1 to 100 for excellence, the lower rungs of the ladder would correspond to poor articles, of various types: stubby, badly written, failing when judged by policy (NPOV, NOR, CYS), non-encyclopedic. This could be the basis of an automated clean-up/deletion mechanism also, but would need perhaps one other ingredient (to make a kind of 2-d plot). What should that be?
Charles
On 07/09/05, charles matthews charles.r.matthews@ntlworld.com wrote:
Trying to think laterally for the moment. If articles were rated on a scale of 1 to 100 for excellence, the lower rungs of the ladder would correspond to poor articles, of various types: stubby, badly written, failing when judged by policy (NPOV, NOR, CYS), non-encyclopedic. This could be the basis of an automated clean-up/deletion mechanism also, but would need perhaps one other ingredient (to make a kind of 2-d plot). What should that be?
Quality of form vs. quality of content?
The one is ranging from an unwikified orphan stub to a polished, linked, well-referenced 5,000 word article article; the other deals with comprehensiveness, NPOV, accuracy, &c.
On 9/7/05, Andrew Gray shimgray@gmail.com wrote:
On 07/09/05, charles matthews charles.r.matthews@ntlworld.com wrote:
Trying to think laterally for the moment. If articles were rated on a scale of 1 to 100 for excellence, the lower rungs of the ladder would correspond to poor articles, of various types: stubby, badly written, failing when judged by policy (NPOV, NOR, CYS), non-encyclopedic. This could be the basis of an automated clean-up/deletion mechanism also, but would need perhaps one other ingredient (to make a kind of 2-d plot). What should that be?
Quality of form vs. quality of content?
The one is ranging from an unwikified orphan stub to a polished, linked, well-referenced 5,000 word article article; the other deals with comprehensiveness, NPOV, accuracy, &c.
I'm certainly not always the best or most dedicated contributor, but whenever I make a significant edit, I try to cite sources and post references. Even when I don't, I make use of the edit summaries to explain my edits.
Since I regularly contribute to Wikipedia, I think it's clear that I don't object to my contributions being re-edited mercilessly.
However, the more I consider these proposed article rating schemes, the more I dislike the idea of my contributions being graded by those who feel it is their right do so without any real feeling of obligation to actually improve the article instead. If these ideas develop in the direction I fear they will (based on my observation of and participation in the various deletion forums), I am certain to decrease or cease my participation.
In other words, I'm willing to put my writing on the table for anyone to re-edit at will. I am far less willing to place it in a shooting gallery for random drive-by snipes to take pot shots at.
I already feel hostility regarding good faith contributions from the deletion forums. There is already significant negativity in the way we treat some new users who do not yet understand how Wikipedia functions. I fear that any article rating forum will almost certainly bring with it further negativity that I'm not interested in.
The Wiki process and participation by many dedicated users assures that the quality of our articles improves over time. I don't see a rating system as essential to the drive toward quality content. In fact, I think it will likely drive users away, decreasing the value of the Wiki process.
Being re-edited, revised, or reverted is more than enough of a "rating" of my work. I can already tell how much the community respects my efforts by the rough proportion of what remains of my contributions months later.
Whatever article rating scheme develops (and there is almost certain to be one, based on the support I see here for one), please take care to prevent it from being a negative experience, especially for new users who frequently aren't ready for or accustomed to the high standards we all strive to achieve.
Michael Turley (michael.turley@gmail.com) [050908 05:06]:
However, the more I consider these proposed article rating schemes, the more I dislike the idea of my contributions being graded by those who feel it is their right do so without any real feeling of obligation to actually improve the article instead. If these ideas develop in the direction I fear they will (based on my observation of and participation in the various deletion forums), I am certain to decrease or cease my participation. In other words, I'm willing to put my writing on the table for anyone to re-edit at will. I am far less willing to place it in a shooting gallery for random drive-by snipes to take pot shots at.
That's a tricky one.
However, people will already hold opinions of the quality of a bit of writing.
The initial idea is to gather the data and not do anything with it (separate the data-gathering and the applications). Just see what the numbers say.
We're a top-50 website - having millions of random readers clicking a "Rate this article" tab and giving their perceptions is, to me, inherently interesting. If writing I've contributed substantially to gets low ratings from LOTS of people, then I fear I must wear such critique from the people we are after all writing this thing for and see how I may improve myself ...
- d.
You think so? That's usually when I brand myself as something separate from the "unappreciative masses"; elitism as a coping mechanism. ;-)
FF
On 9/7/05, David Gerard fun@thingy.apana.org.au wrote:
We're a top-50 website - having millions of random readers clicking a "Rate this article" tab and giving their perceptions is, to me, inherently interesting. If writing I've contributed substantially to gets low ratings from LOTS of people, then I fear I must wear such critique from the people we are after all writing this thing for and see how I may improve myself ...
- d.
David Gerard wrote:
We're a top-50 website - having millions of random readers clicking a "Rate this article" tab and giving their perceptions is, to me, inherently interesting.
This is really interesting. How about something like, "Did you find what you were looking for?"
Wikipedia isn't a service, really, but maybe it wouldn't hurt to think of it as one. Right now, I think it's possbile that Wikipedia editors are more in touch with the issues that affect editors and totally out of touch with people who actually use the damn thing as a reference. All this debate about VfD/AfD or what is and isn't a speedy or whether [[WP:BEARD]] should be moved to BJAODN is probably not what your average Wikipedia reader is thinking about. More reader feedback might be a way to give us an idea of how good a job we are doing.
- Ryan
Ryan Delaney wrote:
David Gerard wrote:
We're a top-50 website - having millions of random readers clicking a "Rate this article" tab and giving their perceptions is, to me, inherently interesting.
This is really interesting. How about something like, "Did you find what you were looking for?"
http://meta.wikimedia.org/wiki/En_validation_topics
Just needs to be turned on...
Magnus
Spot on.
But theres also real issue of ranking particular edits, relative to others. IAUI, this is a lot more processor and data intensive - a lot of metadata can be applied or drawn from each article.
SV
--- charles matthews charles.r.matthews@ntlworld.com wrote:
Ed Poor wrote
*We* must decide that we want some means of
assuring readers that they are getting reliable information.
Simple - just protect decent, informative versions of pages. True, this turns a wiki into a standard static web site; but hey, people will be mightily assured.
I'm sort of amazed that the category system has't already been adapted to give some sort of 'marks out of 10' to articles. If there were real community pressure to rate articles, I think that might have happened already. What I see is only the skeleton of such as system: stub tags, featured-article status, and the reprehensible use of POV tags as comment rather than trying to sort out disputes.
Trying to think laterally for the moment. If articles were rated on a scale of 1 to 100 for excellence, the lower rungs of the ladder would correspond to poor articles, of various types: stubby, badly written, failing when judged by policy (NPOV, NOR, CYS), non-encyclopedic. This could be the basis of an automated clean-up/deletion mechanism also, but would need perhaps one other ingredient (to make a kind of 2-d plot). What should that be?
Charles
WikiEN-l mailing list WikiEN-l@Wikipedia.org To unsubscribe from this mailing list, visit: http://mail.wikipedia.org/mailman/listinfo/wikien-l
______________________________________________________ Click here to donate to the Hurricane Katrina relief effort. http://store.yahoo.com/redcross-donate3/