[Foundation-l] The 'Undue Weight' of Truth on Wikipedia (from the Chronicle) + some citation discussions

[Achal Prabhala]

On Wednesday 22 February 2012 03:45 PM, Thomas Morton wrote:
>> Jokes aside :) the problem here is exemplary of what Wikipedia *doesn't*
>> do well, which is to find ways to assess the legitimacy of
>> not-yet-legitimised knowledge
>
> I'm not seeing a good argument that we *should* assess the legitimacy. This
> seems to be being cast in the light of "verifiability not truth" (a really
> silly maxim) but, in reality, it goes more back to our idea of "we use
> reliable sources because they are *peer reviewed*".

Well actually, we use newspaper sources very frequently, as well as 
non-scholarly (and therefore non-peer-reviewed) books, so in fact, we 
rely on *printing* (or to put it more kindly, publishing) as a signal 
for peer-review, not peer-review itself. In my opinion, this is a poor 
signal.

>
> The implicit suggestion here is that Wikipedia could/should act as that
> form of peer review for so called "not-yet-legitimised knowledge".
>
> Although it would be nice to have that role it isn't actually all that
> practical for several reasons:
>
> - We already have enough disagreement over sourcing as it is
> - Very few of us are truly subject matter experts
> - Even fewer of us have experience of peer review and critical examination
> of work (this is especially critical in the sciences)
> - Taking on the role of peer review puts us at odds with our main aim; of
> providing a summary resource.
>
> The main thing it would do is open up Wikipedia as an avenue to push (and
> legitimise) fringe material.

I completely agree that we need a system that doesn't throw a spanner in 
the works - but if you're suggesting that the only workable signal for 
legitimacy is printing, then that seems odd; and it is odder that while 
we ourselves rely on a range of filtered non-printed sources for our own 
information (social media, conversations) that we shouldn't attempt to 
find a way to bring Wikipedia into these very old and very new systems 
of legitimate knowledge that we've fundamentally accepted ourselves.

>
>
>> whether the 'truth' is new analysis backed up by serious scholarship (as
>> in this case), or things that have not yet made it to reliable print
>> scholarship (knowledge that's circulated orally, whether in conversations
>> or social media). The core of the problem would appear to be our insistence
>> on the narrowest and smallest possible definition of 'legitimate
>> knowledge'.
>
> Is it? Lets look at what happened here.
>
> - Someone posted information apparently based on their own analysis - it's
> not unreasonable to remove this
> - He began to defend his additions on the talk page and some were
> incorporated
> - He gave up further attempts
> - The next day a lot of those comments were incorporated (if you read
> through the detail very carefully, to as much of an extent as the published
> literature allowed) based on the inconsistencies he raised
> - He went away and wrote a book which forwards a number of new theories and
> updates our understanding of the topic.
>
> Has anyone actually read through the points raised? The problem is not a
> case of "well this factual thing disproves what is in the article". It is
> much more a case of disagreement over the established *interpretation* of
> events and over the *extent* to which views expressed by the previously top
> level source were recorded (for example; "no evidence" was a mistaken
> summary of the view raised by the source, a point which was then corrected).
>
>
>> And I'd imagine that the solution is to find a workable, sensible and
>> cross-culturally translatable version of legitimacy that is a lot better,
>> bigger and more generous than what we have.
>
> No it isn't.
>
> We have a good sourcing policy; one which does cover a very wide range of
> sources and can be relaxed and restricted as required to fit the topic
> based on good editorial judgement.

Consider these two points:

1) Bad behaviour needs a back-up, and inadequately updated/ incompletely 
thought out policies serve as a bulwark against weeding out bad behaviour.

2) If, for instance, 'no original research' was to keep physics cranks 
out, as seems the case, then it's succeeded - the physics cranks are 
out. Given that it was put in place ten years ago though, and given that 
it may have been very useful circa 2001, in a Wikipedia with limited 
geographical contribution and use, things are very different now. Might 
we not benefit from assessing the cost of policies that guard against 
enemies who no longer exist?

>
> However, for the topic of *history* (in which I have an interest, and where
> I work on articles at the moment) we definitely should stick to well
> reviewed, published material.
>
> What *was* at issue here is how we treat new users; the discussion was
> approached (on the part of our editors) either as a battleground/fight, or
> in a quite patronising way. The issue here was that someone was put off
> from raising the issues.
>
> I do know of academics who are frustrated by what they see
> as inaccuracies in Wikipedia articles; and when they try to correct them
> from their own knowledge get reverted. That, coupled with a lack of
> understanding of how Wikipedia works from a technical perspective, can make
> the experience very frustrating - and the opportunity to explain the
> rational viewpoint (i.e. peer reviewed sourcing) is lost.
>
> If you read the article this is what he is saying; that academics should
> follow the peer review route before trying to get their material
> included. He also notes that even when he had taken this route he was put
> off because of his treatment the last time.
>
> The failure here is *not* our content policy. But the behavioural.

I respect where you're coming from and it's very helpful in furthering 
my own understanding of the situation. But: I think the 'behavioural' is 
distinctly affected by 'policy' - especially when the policy is 
malleable, loose and archaic enough to interpreted (usually hawkishly) 
at will by those already in the know.

>
> Tom
> _______________________________________________
> foundation-l mailing list
> foundation-l at lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>