Re: [Wikimedia-l] Is Wikipedia Really 90% Wrong

The journal article by Hasty et al published on May 1st 2014 basically took ten Wikipedia articles and ten “researchers” (either medical students or residents). Each Wikipedia article was then assessed by two of these researchers to try to determine how many statements of fact they contained. The first issue was that the number of statements of fact each reviewer found sometimes differed by nearly 100%. They than took these individual facts and the “researchers” compared them with the peer reviewed literature as found on pubmed or the medical website Uptodate. They did not check to see if the sources Wikipedia was using were high quality or were accurately reflected. Additionally medical students and residents are hardly experts in medical research. No errors in Wikipedia are mentioned directly in the original journal article. When I spoke with the lead author he declined to release the underlying data for us at Wikipedia to correct the “errors” they had found stating that he may 1) wish to publish more on the topic and 2) wished to protect the researchers. So much for independent verifiability in science. Hasty did make some claims to the popular press about errors on Wikipedia. Some of the facts he mentioned however accurately reflected some of the best available peer reviewed sources. For example he claimed that blood pressure should only be checked twice to make the diagnosis of hypertension and that when we state three times we are wrong. However look at the National Institute for Health and Care Excellence (previous known as the National Institute of Clinical Health / NICE) on page 7 in this document http://www.nice.org.uk/nicemedia/pdf/CG18background.pdf It is thus a little ironic that the Telegraph, a UK paper, repeated this incorrect statement and the BBC covered the story so uncritically. Wikipedia has strong recommendations for what counts as a suitable source. We recommend the use of secondary sources published in well respected journals from the last 3-5 years, position statements of national or internationally recognized medical bodies or major textbooks. Is Wikipedia a perfect source? No, but it is just as good as many and better than most other sources out there. Or else why would the world be using it? Hasty's work did not have a comparison group. Basically he invented a new method to test the quality of medical content and then only applied this new method to one source, Wikipedia. Without a comparator this single data point is meaningless. I am curious what he would have found if he would have applied this to a NICE guideline or emedicine? We recently surveyed our top contributors and asked about their backgrounds. What we found was that 52% have either a masters, PhD, or MD. Another 33% have a BSc. About half are health care providers. 82% are male, 9% are female and 9% classified themselves as other or would rather not say. This is very similar to results published by Nusa Faric in her master's thesis. Additionally we are working with a number of organizations including: the National Institute of Health, the Cochrane collaboration, and the UCSF college of medicine among others to improve Wikipedia’s health care content. What Hasty did show was 1) the peer reviewed literature does not agree with itself (ie different peer reviewed sources come to different conclusions which is no surprise to anyone that has read much of it) 2) the peer review process is sometimes flawed as he was able to publish a "peer reviewed" article whose data does not support its conclusions. As someone who has read a lot of the peer reviewed literature this is also not surprising. -- James Heilman MD, CCFP-EM, Wikipedian The Wikipedia Open Textbook of Medicine www.opentextbookofmedicine.com _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l(a)lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>