Wow.
Wil - you're going to love WikiData.
Phoebe: I have seen that list of peer-reviewed articles related to Wikipedia medical content. I've extracted those related to quality and added more from a couple of database searches I did in January and the list of 42 (some are letters and there's a conference abstract, though) are collapsed on the WikiProject Medicine talk page now under the heading, "This thread is notable."
I've read most but not all of those and, as Andreas mentioned, most of those suffered from small sample size and poor or opaque sample selection criteria.
Erik, thank you for pointing to the "reviewer" trial. I had read it before and I'm glad to have this opportunity to tell you how much I love it. There is a big hole in Wikipedia where expert reviewing belongs.
I'm presently on the board of WikiProject Med Foundation, but will be stepping down after Wikimania. I mostly edit medical content. Anne is right, it is heavily curated. But stuff slips through the net of patrollers from time to time, and barely a day goes by without some howler of a long-term problem coming to light.
I would like to know - know, rather than rely on my gut feeling - how accurate our medical content is. To know that, I think the first step would be to get an expert on scientific study design to review the 30-40 existing studies that address the quality of our medical content, and tell us what, if anything, we can take from that prior work - essentially what Anne recommends above, but rather than making my own incompetent and heavily biased assessment, get an expert to do it.
My own, inexpert, belief is that those studies are (mostly) so hopelessly flawed that nothing can seriously be generalised from them. If I'm right, I'd then like us all to consider seriously doing a survey whose design is sufficiently rigorous to give us an answer.
Thanks for your thoughts and attention everyone.
Anthony Cole http://en.wikipedia.org/wiki/User_talk:Anthonyhcole
On Thu, May 8, 2014 at 11:05 AM, Andreas Kolbe jayen466@gmail.com wrote:
On Thu, May 8, 2014 at 3:41 AM, Risker risker.wp@gmail.com wrote:
Yes, of course readability analysis is done by automation. I've yet to find a consistent readability assessment that doesn't use automation.
It's
not an area where subjectivity is particularly useful.
And that was an average of 18 minutes per article, i.e., 36 minutes: 18 minutes for the WP article and 18 minutes for the PDQ article. How long
do
you really think it should take? I read several of the articles in
under 5
minutes on each site. Of course, the reviewers wouldn't need to look up the definitions of a lot of the terms that lay people would need to look at, because they were already professionally educated in the topic area,
so
that would significantly reduce the amount of time required to assess the article.
It took me more than 18 minutes to write the last e-mail in this thread. :)
The lung cancer article, for example, which was among those reviewed, has well over 4,500 words of prose, and cites 141 references. That's a reviewing speed of 250 words per minute. I don't know if you have ever done an FA review ...
Andreas, you seem to have pre-determined that Wikipedia's medical
articles
are all terrible and riddled with errors.
And I think you are being needlessly defensive. I have an open mind as to what the results might be. What I am sure of is that neither you nor I nor the Foundation really know how reliable they are. Why not make an effort to find out?
Realistically, they're amongst the most likely to receive professional editing and review - Wikiproject Medicine does a much better job than people are willing to credit them.
Yes, and many editors there are sorely concerned about the quality of medical information Wikipedia provides to the public.
Incidentally, there was a discussion of the JAOA study in The Atlantic today:
http://www.theatlantic.com/health/archive/2014/05/can-wikipedia-ever-be-a-de...
A member of WikiProject Medicine is quoted in it, as is the study's author.
—o0o—
So both sides acknowledge: There are errors in Wikipedia’s health articles. And that’s a problem, because people use them.
—o0o—
The biggest weakness to the articles - and I've heard this from many
people
who read them - is that they're written at too high a level to be really accessible to lay people. I thought the point that the study made about the benefit of linking to an "English" dictionary definition of complex terms rather than to another highly technical Wikipedia article was a
very
good one, for example. We could learn from these studies.
Indeed, many science articles are mainly written by professionals in the field (I noted math and physics earlier, but chemistry and of course a large number of computer articles are also written by professionals).
The
biggest challenge for these subjects is to write them in an accessible
way.
Note, I said "science" - alternative medicine, history, geopolitical and "soft science" articles are much more problematic.
Risker/Anne _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe