Folks,
The LA Times health blog Booster Shots reports:
http://latimesblogs.latimes.com/booster_shots/2010/06/three-cheers-for-wikip...
As it turns out, information on Wikipedia can largely be trusted, at least as it pertains to cancer. That should be a relief both to patients and to the doctors who care for them. The entries in the user-edited online encyclopedia often show up high atop search-engine results, and many users likely have taken their content at face value.
But that content's reliability has been in doubt. After all, it's created by users, not traditional "experts." ("Don't use Wikipedia," earnest eighth-graders in search of homework help are told.)
Now researchers at Kimmel Cancer Center at Thomas Jefferson University in Philadelphia have done their own analysis of that content, comparing Wikipedia information on 10 types of cancer to information found in the National Cancer Institute's Physician Data Query.
The entries were solid, the researchers found, at least in terms of key points. Way to go, online writers and editors! But they were also quite dense. Tsk -- points subtracted due to lack of clarity, online writers and editors.
The researchers write in their study's abstract, to be presented at the current annual meeting of theAmerican Society of Clinical Oncologyhttp://chicago2010.asco.org/: "Although the Wiki resource had similar accuracy and depth to the professionally edited database, it was significantly less readable. Further research is required to assess how this influences patients' understanding and retention."
Here's the abstract of the Wikipedia analysishttp://abstract.asco.org/AbstView_74_41625.html; one of the researcher's commentshttp://www.jeffersonhospital.org/News/2010-june-cancer-information.aspx, as presented in the university's news release; and the aforementioned Physician Data Query http://www.cancer.gov/cancertopics/pdq, a peer-reviewed cancer database.
Surely, no one needs help finding Wikipedia. But here's how it's createdhttp://en.wikipedia.org/wiki/Wikipedia:About, worth reading now more than ever.
http://abstract.asco.org/AbstView_74_41625.html
The abstract of the analysis is here:
Regards
*Keith*
The problem is the fundamental issue of rapidly changing content; a snapshot analysis will never give you a good grasp of an article (or all of Wikipedia's) general reliability, because any article can be perfectly accurate in one minute and horribly misleading in another. Any article about Wikipedia's reliability as a source for key information should have that as a caveat.
Nathan
On 1 June 2010 23:06, Nathan nawrich@gmail.com wrote:
The problem is the fundamental issue of rapidly changing content; a snapshot analysis will never give you a good grasp of an article (or all of Wikipedia's) general reliability, because any article can be perfectly accurate in one minute and horribly misleading in another. Any article about Wikipedia's reliability as a source for key information should have that as a caveat.
We often state this, but I'd be intrigued to know how true it is on average. Have there been any recent studies on the volatility of the quality of (non-current-event) articles?
Keith Old wrote:
The researchers write in their study's abstract, to be presented at the current annual meeting of theAmerican Society of Clinical Oncologyhttp://chicago2010.asco.org/: "Although the Wiki resource had similar accuracy and depth to the professionally edited database, it was significantly less readable. Further research is required to assess how this influences patients' understanding and retention."
So instead of getting an overall bird's eye view of a subject, we end up with a nerd's eye view.
Ec
On 02/06/2010, Ray Saintonge saintonge@telus.net wrote:
So instead of getting an overall bird's eye view of a subject, we end up with a nerd's eye view.
All that's happened is that the professionally produced material had some specific attention towards making it readable.
The Wikipedia AFAIK doesn't have any formal processes to check that, so far as I know.
There are reading age metrics so it might be well worth running a bot over the whole wiki to calculate them and add them to the talk page to help people work out whether articles are written at an appropriate level for their content.
Ec
On Wed, Jun 2, 2010 at 2:03 AM, Ian Woollard ian.woollard@gmail.com wrote:
All that's happened is that the professionally produced material had some specific attention towards making it readable.
The Wikipedia AFAIK doesn't have any formal processes to check that, so far as I know.
Is it not a criterion used when judging articles C/B/A/GA/FA?
User:Bodnotbod
Bod Notbod wrote:
On Wed, Jun 2, 2010 at 2:03 AM, Ian Woollard ian.woollard@gmail.com wrote:
All that's happened is that the professionally produced material had some specific attention towards making it readable.
The Wikipedia AFAIK doesn't have any formal processes to check that, so far as I know.
Is it not a criterion used when judging articles C/B/A/GA/FA?
Our processes are unlikely to pick up the most obvious difference (as I judge from an example), namely that where we would wikilink a technical word, the NCI would give a phrase of definition in parentheses beside it. We think people who need to know what the [[colon]] is will click and find out, they don't use indirection in that way.
Charles
I think the study does an excellent, if only implicit, job of picking up a growing thread about Wikipedia quality, and one that I have often observed in my own research of Wikipedia. Professionally-written articles (in this case on cancer) are very clearly and explicitly written with the education of the general reader in mind. Terms are carefully explained and care is taken to ensure that the material is easy to understand. Such articles are, generally, self-contained as Charles Matthews points out: you don't have to click through to find out what unfamiliar words mean. Citations are not included, as they are irrelevant to the main purpose, though further reading is often given.
Wikipedia, on the other hand, has evolved from being geared at the general reader to being a form of, if you'll pardon by somewhat rude phrase, a bastardized journal article. While professional health information (by which I mean professional general-interest health information) doesn't give citations, Wikipedia articles (for reasons that are perfectly logical) are full of them. While professional health information avoids the use of technical terms for readability, Wikipedia tends to use them in favor of comprehensiveness. An excellent piece of professional health information provides the reader with everything he wants to know, but would offer little to the professional. Wikipedia, on the other hand, tries to offer something that would interest the professional.
The problem is that experts just aren't interested in reading Wikipedia articles (within their area of expertise). If a researcher of physician wants up to date cancer information, he will look at the relevant books and journal articles. Recent, interesting research on Australian medical students found that while early in their school careers, the students often use Wikipedia, its use declines precipitously as they gain more experience (and I think we can all agree that's a good thing). To put it simply, Wikipedia will NEVER be able to compete with what is published in technical books and journals in any field of academic inquiry.
So, then, why are we trying? Why do the "best" Wikipedia articles look more and more like (poorly done) journal literature reviews full of technical terms and requiring substantial background knowledge to understand? I, for one, despite several years of college mathematics find nearly all math articles largely incomprehensible because they are clearly not aimed at the general reader. But, the general reader IS Wikipedia's audience, and we should write the articles that best serve him.
David Lindsey
On Wed, Jun 2, 2010 at 11:44 AM, Charles Matthews < charles.r.matthews@ntlworld.com> wrote:
Bod Notbod wrote:
On Wed, Jun 2, 2010 at 2:03 AM, Ian Woollard ian.woollard@gmail.com
wrote:
All that's happened is that the professionally produced material had some specific attention towards making it readable.
The Wikipedia AFAIK doesn't have any formal processes to check that, so far as I know.
Is it not a criterion used when judging articles C/B/A/GA/FA?
Our processes are unlikely to pick up the most obvious difference (as I judge from an example), namely that where we would wikilink a technical word, the NCI would give a phrase of definition in parentheses beside it. We think people who need to know what the [[colon]] is will click and find out, they don't use indirection in that way.
Charles
WikiEN-l mailing list WikiEN-l@lists.wikimedia.org To unsubscribe from this mailing list, visit: https://lists.wikimedia.org/mailman/listinfo/wikien-l
On 2 June 2010 12:42, David Lindsey dvdlndsy@gmail.com wrote:
So, then, why are we trying? Why do the "best" Wikipedia articles look more and more like (poorly done) journal literature reviews full of technical terms and requiring substantial background knowledge to understand? I, for one, despite several years of college mathematics find nearly all math articles largely incomprehensible because they are clearly not aimed at the general reader. But, the general reader IS Wikipedia's audience, and we should write the articles that best serve him.
It's because ability to look up and cite facts is a lot more common than actually being a good writer. There's little cure for this apart from actual good writing being applied.
The "best" articles are the creation of algorithmic and judgement-impaired FA/GA review processes. You get what you measure. How to measure good writing?
Personally I would prefer an article to have all the details on a subject and imperfect lumpy writing than be polished with details smoothed away. Wikipedia is a work in progress, and I see nothing wrong with that being visible. But that's just me, I wouldn't generalise it to everyone.
The easy thing concerned Wikipedians could do is at least make sure article summary intros are well-written and concise without losing important detail - remember that the lead should ideally be a standalone short article in itself. (The mobile gateway presents articles this way by default, for instance.)
That said, sometimes I'm freshly amazed by this thing we've built. I looked up [[Betelgeuse]] yesterday (because of the rumours that it was about to finally go supernova [*]) and, of course, found myself with about thirty tabs on supernovae, giant stars, neutrinos, why neutrinos have mass ... it was all *really good*, *impeccably referenced* and *up to date*. Some of the writing was clunky, but I'm enough of an editor and popular science fan to have been able to untangle bits. This Wikipedia thing - it's often really very good indeed, you know.
- d.
[*] Bad Astronomy: Is Betelgeuse about to blow? http://blogs.discovermagazine.com/badastronomy/2010/06/01/is-betelgeuse-abou...
David Gerard wrote:
Personally I would prefer an article to have all the details on a subject and imperfect lumpy writing than be polished with details smoothed away. Wikipedia is a work in progress, and I see nothing wrong with that being visible. But that's just me, I wouldn't generalise it to everyone.
We should undoubtedly stick to doing one thing well. And our "thing" does appear to be collation. I'm happy for WP's cancer coverage to make it into the same sentence as the NCI's. It argues that some very serious work has gone on by well-informed people. For all our guidelines, I think enWP does not emphasise writing well; but then trying to bring up the minimum standard of articles by the thousand has been more important historically (and still is, as far as I can see).
Charles
On 2 June 2010 14:10, Charles Matthews charles.r.matthews@ntlworld.com wrote:
We should undoubtedly stick to doing one thing well. And our "thing" does appear to be collation. I'm happy for WP's cancer coverage to make it into the same sentence as the NCI's. It argues that some very serious work has gone on by well-informed people. For all our guidelines, I think enWP does not emphasise writing well; but then trying to bring up the minimum standard of articles by the thousand has been more important historically (and still is, as far as I can see).
Yeah. But making intros very clear on prominent articles shouldn't be too laborious. Care, attention, talk page note, etc. Start using it as more of a tick-box point at article reviews, perhaps.
Wikipedia articles tend to accumulate cruft with time - special case subclauses, trivial POVs demanding a sentence and so on. Decruft occasionally and things will be fine.
- d.
On Wed, Jun 2, 2010 at 1:22 PM, David Gerard dgerard@gmail.com wrote:
The "best" articles are the creation of algorithmic and judgement-impaired FA/GA review processes. You get what you measure. How to measure good writing?
What do you mean by algorithmic?
And what do you feel needs changing in the review processes?
User:Bodnotbod
On 2 June 2010 15:27, Bod Notbod bodnotbod@gmail.com wrote:
On Wed, Jun 2, 2010 at 1:22 PM, David Gerard dgerard@gmail.com wrote:
The "best" articles are the creation of algorithmic and judgement-impaired FA/GA review processes. You get what you measure. How to measure good writing?
What do you mean by algorithmic?
I think I mean more like "procedural". There's a list of criteria. These are what is measured so these are what are optimised for.
And what do you feel needs changing in the review processes?
FAs are frequently all but unreadable to the casual reader. How feasible would it be to add "intro clear to casual reader"? I realise some topics are just never going to be that clear ... particularly with the tendency for FAs to be about specialised topics.
- d.
On 2 June 2010 18:00, David Gerard dgerard@gmail.com wrote:
FAs are frequently all but unreadable to the casual reader. How feasible would it be to add "intro clear to casual reader"? I realise some topics are just never going to be that clear ... particularly with the tendency for FAs to be about specialised topics.
And I realise I've just said "hey, let's add another rule!" which is an intrinsically bad thing. So if someone can come up with another idea then that would be really good.
- d.
On 06/02/2010 10:01 AM, David Gerard wrote:
FAs are frequently all but unreadable to the casual reader. How
feasible would it be to add "intro clear to casual reader"? I realise some topics are just never going to be that clear ... particularly with the tendency for FAs to be about specialised topics.
And I realise I've just said "hey, let's add another rule!" which is an intrinsically bad thing. So if someone can come up with another idea then that would be really good.
Can we just test the readability to the casual reader? No rules or anything; just make the data available and trust that people will use it wisely.
I could imagine a few different approaches to that:
1. On every page we put a feedback widget. A simple three-clicks-and-done thing, with the option to offer more detail. 2. Randomly sampling readers, where we ask 1 reader in X for their opinion on the article. 3. A separate article evaluation tool, somewhat like fivesecondtest.com. Anybody can nominate a page for evaluation. Volunteer readers are assigned random pages. They give feedback. Maybe, to see if the basic ideas come across, they also summarize what they remember after the page is closed. 4. Something like Mechanical Turk, where we test different versions of the same paragraph or section to see which works better with readers.
All of these have their issues, and I'm sure there are others, but basically I'm saying that we could use our vast traffic and the immense goodwill of our readers to get real data on reader experience.
William
Yes, Intro to X articles would be nice. There are a handful floating around, such as http://en.wikipedia.org/wiki/Introduction_to_general_relativity, but often attempts to create such articles are criticized as content forks, which is unfortunate.
On Wed, Jun 2, 2010 at 6:00 PM, David Gerard dgerard@gmail.com wrote:
FAs are frequently all but unreadable to the casual reader. How feasible would it be to add "intro clear to casual reader"? I realise some topics are just never going to be that clear ... particularly with the tendency for FAs to be about specialised topics.
- d.
WikiEN-l mailing list WikiEN-l@lists.wikimedia.org To unsubscribe from this mailing list, visit: https://lists.wikimedia.org/mailman/listinfo/wikien-l
On 2 June 2010 18:51, David Lindsey dvdlndsy@gmail.com wrote:
On Wed, Jun 2, 2010 at 6:00 PM, David Gerard dgerard@gmail.com wrote:
FAs are frequently all but unreadable to the casual reader. How feasible would it be to add "intro clear to casual reader"? I realise some topics are just never going to be that clear ... particularly with the tendency for FAs to be about specialised topics.
Yes, Intro to X articles would be nice. There are a handful floating around, such as http://en.wikipedia.org/wiki/Introduction_to_general_relativity, but often attempts to create such articles are criticized as content forks, which is unfortunate.
The sharp end of physics has special qualities:
1. Our articles on it are very good and very up-to-date (as I noted). 2. Even the obscure stuff is utterly undeniably encyclopedic and we have lots of high-quality sources. (Even the cutting-edge discourse - arXiv preprints and physics blog posts - is good enough for many of our purposes, particularly as backgrounders on the abstruse technical peer-reviewed papers.) 3. Actually understanding it is beyond almost anyone reading. (My maths sputtered to a halt in the middle of second-year engineering.) But the overviews are sufficiently comprehensible and quite fascinating.
So intros are very clearly reader-useful, and procedural types can be asked why both can't be kept ;-)
Perhaps intro articles could start in similar high-science fields.
It would be *ideal* for both to be a single article, but that would I suspect lead to unwieldy novel-length articles.
- d.
On Wed, Jun 2, 2010 at 6:00 PM, David Gerard dgerard@gmail.com wrote:
FAs are frequently all but unreadable to the casual reader. How feasible would it be to add "intro clear to casual reader"? I realise some topics are just never going to be that clear ... particularly with the tendency for FAs to be about specialised topics.
I reviewed our article on [[Lemurs]] for FA. I told the main contributor that it would be excellent if he would put an article on Simple Wikipedia and he said he had plans to do so.
I intend to carry on reviewing FAs, so I'll bear that in mind for when I give feedback.
On Wed, Jun 2, 2010 at 7:53 AM, Keith Old keithold@gmail.com wrote:
The researchers write in their study's abstract, to be presented at the current annual meeting of theAmerican Society of Clinical Oncologyhttp://chicago2010.asco.org/: "Although the Wiki resource had similar accuracy and depth to the professionally edited database, it was significantly less readable. Further research is required to assess how this influences patients' understanding and retention."
Does this signal some advance in public perception of Wikipedia? At last we leave behind the question "is it accurate", and move on to "is it well written"?
Steve