[WikiEN-l] WP:RS rewrite proposed

[Phil Sandifer]

I've posted a detailed critique of RS to the talk page. I'm reposting  
it here (still in wikiformatting) to make sure this issue gets the  
wide attention it deserves, and because the nature of a talk page is  
that it's going to become unreadable in 20 minutes or so.

===A somewhat interminable list of flaws===

As requested, an inventory of the flaws in this page. I hope you will  
see that the flaws are deeply rooted, affecting the organization and  
goal of the page. Not only is this an inadequate guideline for  
sourcing, it is an inadequate foundation for a guideline. David is  
correct - this needs to be rewritten from the ground up.

There are three basic flaws of this page, and they'll recur  
throughout the below.

#The page arbitrarily attempts to rule out subjective judgment in  
some cases while mandating it in others. The cases where it mandates  
it are often masked as didactic guidelines that depend on phrases  
like "reliable X," "common Y," or a mushy definition of fact. These  
phrases could be defined in the same way that "reliable source" has  
been, but such definitions would necessarily run into the same  
problems of subjectivity. It's turtles all the way down.
#The page is, at numerous points, clearly written for a narrow range  
of topics. When applied to other topics, it ranges from the merely  
unhelpful to the completely wrong.
#Sections flatly contradict each other.

More details follow.

====Opening====

The opening section in general and paragraph three in particular  
frame the page in an astonishingly flimflam way. The prospect of  
unsourced information being removed is raised, but no serious  
suggestion is given as to when "it is better to have no information  
at all than to have information without sources." The result is to  
give a vague and nonspecific warning when, in fact, a specific  
warning is in order. We are, after all, talking (at least primarily)  
about BLP here.

====Definitions====

The definition of fact is too iron-clad, relying ultimately on the  
idea that nobody "seriously disputes" a claim. By this standard,  
neither evolution nor global warming are facts. No serious  
encyclopedia should assert that evolution is not fact, but this  
definition of fact leads us inexorably to that conclusion.

The problem is exacerbated in the next definition, whereby opinions  
are yolked to verifiability, or lack thereof. This contrast between  
fact and opinion leaves a vast no man's land of verifiable  
information that people still disagree with. Worse, this no man's  
land is not self-evident. Does the statement "The US war in Iraq was  
conducted based on false information regarding the presence of WMDs  
in Iraq" count? It's verifiable, but it's not universally held.

To be clear, THERE IS NO WHITE LINE DEFINITION of fact and opinion  
that will let us automatically tell if a statement should be phrased  
as fact or as "X thinks Y." There are white line cases - Mars is a  
planet, Christians believe that Jesus is the son of God. But there's  
a vast middle ground that needs to be taken case by case.

Onward, the definitions of sources are based on old language, but  
have been altered past the point of usefulness. The biggest problem  
is the idea that primary sources require a reliable publisher. Here  
the reliable publisher is being used as a sort of surrogate secondary  
source - we're not using the primary source, but rather the primary  
source under the imprimatur of Publisher X. All of this is justified  
under the idea that "most primary-source material requires training  
to use correctly." (An idea that is untrue. They require care to use  
correctly. Training is often given to help people become more  
careful, but it is not the training that is required. There is a  
difference between using historical archives and running a nuclear  
reactor.)

The problem is that the same can be said of secondary sources. Even  
ones published by scholarly presses, which are, oddly, the only  
secondary sources endorsed, creating a system whereby we are bound  
entirely to scholarly sources on a topic. (Thankfully, the page is  
not consistent enough to endorse that position throughout) Secondary  
sources require just as much care as primary sources, and the  
privledging of them is nonsense.

The original statement on primary sources from which all of this  
derived stated that it's original research to organize primary  
sources in a "novel" fashion. But it never set up such a wide-ranging  
ban on primary sources, and with good reason.

Finally, the distinction makes no light of the fact that the primacy  
of sources is contextual. An example of just how bad this can get:  
There is a poem by Yeats called "Among Schoolchildren." The literary  
critic Paul de Man has a famous reading of this poem that has been  
published by scholarly presses. This reading is a secondary source in  
terms of the Yeats poem. But despite its scholarly status, it's  
tremendously contested, since de Man is a controversial  
deconstructionist - widely recognized as brilliant and important, but  
not always agreed with. ''Furthermore'', de Man's reading is a  
secondary source ONLY on the topic of Yeats. It's a primary source on  
the topic of de Man. And due to the nature of the field, secondary  
sources on de Man's reading are virtually all critical - not because  
his reading does not have adherents, but because publishing an  
article reconfirming de Man's opinion is not considered a useful  
publication, and so only contrary opinions get published.

This is neither an overly convoluted example, nor an unusual one.  
There is an endless list of topics that require this level of thought  
to untangle the nature of primary and secondary sources.

The hedging about using other encyclopedias is a pleasant and rare  
example of actually molding the guideline to reality, but is done in  
a uselessly clumsy way, amounting to "unsigned articles in  
encyclopedias aren't good enough, but we use them anyway." The result  
moves towards a hardline sourcing guideline with a thousand  
asterisked exceptions. This is not a direction in which a usable  
guideline can be found.

====Unattributed material====

The instruction not to remove material that you believe to be true  
and common knowledge is important, but too weak. The reference to  
[[Wikipedia:Common knowledge]] is distressing, both because it's  
unclear to me that the example given (Earth's elliptical orbit)  
satisfies the criteria there, and because that page seems to me to  
have a section that tries to depricate [[WP:AGF]] in favor of  
wikiquette. This use of common knowledge is equivalent to the fact/ 
opinion mess above - a situation where there is no white line is  
being phrased as though there were a white line. In reality, the  
judgments over what does and doesn't need a source are largely done  
on a case-by-case basis - not by referring to a definition of common  
knowledge.

This points towards a larger flaw in our understanding of sources. We  
are basically set up to defer almost universally to the person asking  
for a source. In reality, we need to recognize that source requests  
can be made in bad faith or in error. There are many cases where the  
answer to a request for a source is "No." Requesting a source needs  
to not be fetishized as an innately reasonable act. Like adding  
dispute tags or cleanup tags, it is often a reasonable and helpful  
act. It can also be a trolling or stupid act.

====Beware false authority====

This section starts off by demonstrating the problem with valorizing  
secondary sources - the process of vetting them is, in the end,  
subjective, and often no easier than vetting primary sources. The  
sole privledging of academic sources continues here, impovershing  
fields with less academic research. (Popular culture, current events,  
things related to homelife [Washing machines, cooking, furniture],  
non-theoretical aspects of computing, etc).

The discussion of textbooks is valid only for the sciences and to a  
lesser degree the social sciences (Where it is valid basically on an  
undergraduate level only). It is utterly useless for the humanities,  
where few college textbooks exist, and fewer still are assigned.

====Exceptional claims====

"Reputable news media" is yet another case of a deep well of  
subjective judgment masquerading as a white line policy. "Reports of  
a statement by someone that seems out of character, embarrassing,  
controversial, or against an interest they had previously defended"  
is similarly flawed.

====Evaluating sources====

In many fields, it is nigh-impossible to publish a critical review of  
anything. This would be one thing if bad books were simply not  
reviewed, but in practice bad books are badly reviewed, with reviews  
focusing excessively on positive points while ignoring negative points.

This does not, of course, mean that sources in those fields cannot be  
evaluated.

====Check multiple sources====

Certainly nice practice, but is this, practically speaking, ever  
going to happen?

====Issues to look out for====

The Stormfront/Al Qaeda/Socialist Workers Party comment is another  
example of the thousand-exception style of policy writing.

====Independent secondary sources====

This seems, ultimately, to point towards coming up with an absolute  
account. NPOV neither mandates nor encourages this.

====Online Sources: Evaluating Reliability====

The tone of this section is good, but it's trying to collapse all the  
common sense into one section, and still suffers from a bias towards  
academic sources. The main problem here is that, by being collapsed  
into so small a section, it ends up going in circles, and ultimately  
reads more as "Evaluating sources is hard" but doesn't really point  
toward an endpoint.

====OS: Bulletin boards, wikis and posts to Usenet====

This section is just nonsense. Usenet, BBs, and wikis are perfectly  
reliable as primary sources in lots of cases. [[Spoo]] is a featured  
article based almost entirely on BB posts and Usenet, and nobody with  
any knowledge about the topic would criticize a one of them.

====OS: Self-publication====

This policy does not fit sensibly with the previous section. Anyone  
can start a blog, making the identity verification equally  
problematic. This renders the two functionally equivalent. In some  
cases, of course, we can verify a blogger's identity. We can also, in  
some cases, verify a message board poster's identity. Also, the  
professional researcher criteria is insufficient. Consider a  
situation like Ronald Moore, creator of the new Battlestar Galactica.  
He maintains a blog on the Sci-Fi Channel's website. This is clearly  
a reliable source of information about the show, despite the fact  
that he is not a "researcher" in any useful definition.

====OS: Self-publication on self====

The above BSG example (As well as the Spoo example) fail this test  
too. No independent corroboration is going to be forthcoming on such  
examples of information being released about a media product. (How  
would this even be possible? No secondary source is going to base  
itself on anything other than the statements of the creators.) This  
is a BLP guideline masquerading as a larger guideline. A more sane  
guideline would note that when a person's own account is contradicted  
by other accounts, this should be noted - not to cast de facto  
suspicion on a person's own account.

====OS:Self-published sources as secondary sources====

The blanket ban on using self-published sources as secondary sources  
is in flat contradiction with the "professional researcher" clause  
several sections up.

Why is extreme caution necessary to use, say, the Stormfront website  
as a primary source about Stormfront's beliefs? This is silly. What  
we're trying to prevent is allowing Stormfront to write their own  
biography.

The note about company websites is not a matter of reliable sources -  
it's a matter of NPOV. A general note that, when conclusions of  
primary sources are contradicted by other sources, whether primary or  
secondary, we do not defer wholly to the primary source would be both  
sufficient and appropriate.

====Finding good sources====

A bit of a pep talk, and an odd one at that. We should be cautious  
about suggesting that the ideal Wikipedia editor is going to devote  
an inordinate amount of time to the task and go do book-based  
research. It's nice, yes, but it does cut rather severely against the  
notion that "anyone" can edit the encyclopedia, and is an affront to  
the volunteer nature of the project. We should make sure this  
guideline can be followed by the volunteers we have - not by the ones  
we wish we had.

====BLP====

This is not a guideline related to sourcing.

====History====

This section is working more towards a notion of "authoritative"  
sources instead of reliable ones. Authoritative sources are of  
interest only if we are trying to present an absolute point of view.  
We're not.

====Sciences====

The claim about "reporting material in different fields" is  
problematic, due particularly to the rise of interdisceplenary journals.

Multiple studies have pointed to systemic flaws in the scientific  
peer review system. This becomes a serious issue when dealing with  
cutting-edge science, which is often difficult at best to summarize  
in generalizable terms. The combination of these facts poses serious  
problems for verifiability. (This is a case where we need to be  
worried about primary sources - where a non-expert could not possibly  
interpret them.)

The popular press does not cover science less well than any other  
academic field. This warning should not be specific to science.

Unless we want to create a general list of reputable publications,  
the "which science journals" section doesn't actually provide useful  
guidance.

arXiv needs to be used with extreme caution, but an article on, say,  
the Poincare conjecture can't really be written without it at present.

The statistics section encourages exactly the sort of judgment that  
the page forbids on other topics such as blogs and Usenet.

====Popular culture and fiction====

As said before, nonsense - popular culture articles on contemporary  
culture cannot be written without reference to blogs, Usenet,  
bulletin boards, etc. This is a fundamental shift in what a source  
is, and we need to respond to it.

====References====

No reputable style manuals or research guidelines are cited - only  
Jimbo's statements.

====External links====

Again, a subject-specific guideline is being passed off as a general  
guideline. These pages on primary/secondary sources are good for  
history - not for the general case.

That's all I've got. [[User:Phil Sandifer|Phil Sandifer]] 17:29, 10  
October 2006 (UTC)

Best,
Phil Sandifer
sandifer at english.ufl.edu

You are standing in an open field west of a white house, with a  
boarded front door. There is a small mailbox here.

 >


On Oct 10, 2006, at 6:48 AM, David Gerard wrote:

> http://en.wikipedia.org/wiki/Wikipedia_talk:Reliable_sources#Top-to- 
> bottom_rewrite_proposed
>
> WP:RS is an essay, not a guideline. I can say that because the
> incumbents are now blindly edit-warring to their preferred version,
> even against corrections of grammatical errors. That's the sign of a
> process that has become way too introverted and really, really needs
> to be brought back in touch with the real world.
>
>
> - d.
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at Wikipedia.org
> To unsubscribe from this mailing list, visit:
> http://mail.wikipedia.org/mailman/listinfo/wikien-l