[WikiEN-l] Expert feedback on Featured Articles

[Jussi-Ville Heiskanen]

Nihiltres wrote:
> (Pardon me if I'm doing something wrong regarding the mailing list, I'm a newbie to posting despite lurking for some time)
>
> David Lindsey wrote:
>
>   
>> Finally, though this idea failed to gain any real traction on wiki, I would
>> like to state my support for the idea of adding a fifth criterion to
>> WP:WIAFA: "5. The article, if possible, has been reviewed by an external
>> subject-matter expert."  Even if no such criterion is added, though, I would
>> like to emphasize that it will always, or nearly always, be productive to
>> attempt to find an expert reviewer.
>>     
>
> </snip>
>
> Yes, expert reviewers are generally a good thing. I think that there might be some trouble with controversial articles (if the expert is slanted one way or another, a review correcting "mistakes" might not be helpful to NPOV). The dubiousness of Citizendium's homeopathy article makes a good example.
>   
+1

> I don't currently believe that expert review should be added as a criterion for Featured articles, however, because of the classic problems with experts. First,  how would the interactions/credentials be verified? I can just imagine it now: "I'm submitting this article 'Unobtainium synthesis' to FAC because I think it meets the criteria. Also, the article got an excellent rating on review by Dr. L. Olfake, which she emailed to me the other day." (In case you didn't catch it: "lol fake") Second, how do we avoid Citizendium-like problems? (This is certainly an issue, see also the earlier "Citizendium dead?" thread in this list) Third, would it add systemic bias, or extra hurdles to the process, as Charles Matthews mentions? I'll stop touting the issues, because that's not quite helpful, but there are significant problems with integrating expert review that we ought to address (if not solve) before making it remotely mandatory.
>
> If we were to integrate expert review, I'd start it as another *layer* of Wikipedia. I strongly believe that showing very prominently the level of review a given article—or even a given *revision* thereof—has received, and the perceived level of quality involved, is a good thing.  The Wikipedia 1.0 assessment system (Stub, Start, C, B, A, GA, FA…) seems to serve as a decent start for that sort of thing. (In the long run, I'd love to see some system integrating this with some addition to the planned "patrolled revisions" feature that adds passive flags to revisions.) If we were to bolt on an expert review process to the Featured article system, why not make it a new level? I can imagine it now: "FA+". Take a community FA, and give it a (reasonably verified) expert review, and if all outstanding issues are addressed, call it an FA+ instead. Granted, this involves all sorts of instruction creep, but it's just a starting idea. A practical implementation is left as an exercise for the reader. ;)
>
> I think that expert review is a good idea (we like expert participation as long as they're not jerks about it), but I'm cautious about its feasibility in Wikipedia as an "official" process.
>
> </ramble>
>
>   
First let me just say generally pretty much every word in your "ramble"
is genuinely insightful. Let me share some thoughts it inspired in myself.

To me it seems people who could be tapped for outside input to wikipedia,
even when they are people who generally don't want to edit wikipedia
actively themselves, shouldn't be limited to academics by any means.
Think journalists and professional people in their field doing work as an
ordinary day job just as two examples.

I wouldn't limit such input into a single mold either. There could be one
system developed for just quickly checking a single fact and asking if
they could point to an authoritative source which could be cited in the
article. Somebody who knows their shit might have things right handy,
and not mind telling what it is, so long as they don't have to insert it
into wikipedia themselves, and watch over it to make sure it stays put.

Another system might be recruit such folks to give an impartial
summary overview of what problems if any kind a specific article
in their field might have, be it bias, unbalance of coverage, facts
missing, surfeit of inessential information stuffed in, or simple
errors of fact. I do agree that in no way should this kind of system
be married to the FA process, for the reason that I don't think
there is grounds for limiting it to articles on that level, and of
course it would add a new hurdle to the FA process if it was
absollutely mandatory, and new hurdles the FA process doesn't
really need. And necessarily the resulting summary view of the
article could *never* be thought to be genuinely authoritative;
that would just be outright impossible. I think somebody said
of economics as a science that if you ask for an opinion on a
question on their field from 3 experts, you get 7 opinions. The
only way you could remotely make that work is if you had for
each field a full _panel_ of experts doing the reviewing and I
am sure everybody understands how heavy such a system would
be.

One interesting idea though, would be in some far future,
to actually *hire* fact-checkers for that elusive printed on
paper (or other fixed media), which would be genuinely of
a higher consistency than the raw product we daily see and
will continue to forevermore see on our "unending construction
site". I wouldn't hold my breath to seeing it quite next year
though. The money just isn't going to be there soon to hire
people to even attempt that monumental task, in its entirety.


Yours (sorry about the length of the post),


Jussi-Ville Heiskanen