Brion Vibber wrote:
David Gerard wrote:
That's because the maps are marked
"ALPHA QUALITY - NOT FINAL
PRODUCTION VERSIONS" and are written in pencil. Never mind they're in
the top-40 maps in the world - the project has peaked way too early.
Damn we need the rating feature. What's holding it up right now? List
please, referring to current version of code. (I know the servers are
creaking ...)
Keep in mind that the cute little survey thing doesn't have anything to
do with a system for marking particular page revisions as 'public-ready'
or 'approved' in any useful way.
That's an entirely unrelated and separate issue (and one the project
actually needs), which Special:Validate (which might, hypothetically,
produce 'interesting data' of some sort to some one) does nothing to
help directly.
-- brion vibber (brion @
pobox.com)
I disagree. I think it is a mistake to dismiss the article rating system
_a priori_ as not being useful in the effort to create Wikipedia 1.0.
This certainly is not the opinion of many participants on this list.
In my opinion, a Nupedia-style review procedure for every single article
clearly can't possibly scale to rate almost a million articles in any
reasonable timescale -- the painfully slow featured article process, or
the conflict-ridden AfD process are good examples of the limitations of
formal article-rating systems.
However, collective labelling systems certainly can work; look at the
category system, for example, which is almost magically self-organizing,
or Google, which uses simple link graph adjacency to do much of its work.
An article rating system can sort the possible wheat from the obvious
chaff, with a formal review system then being applied to those articles
where there is either a significant spread of article ratings, or where
an editor or external source has explicitly asserted significant
inaccuracy, insufficiency or bias in an article (and category tagging
could certainly be a useful way to indicate this).
By doing this, the formal review process can be applied in the places
it's most needed.
If there are performance problems with writing and reading the database,
why not during the initial blind rating period simply write the ratings
using syslog over UDP, and allow (possibly anonymized) versions of these
flat-text logfiles to be downloaded by whoever is interested in
analyzing the results? Comma-separated variable files with one line per
entry would do just fine, and the occasional loss of a rating due to
packet loss would matter very little, providing that it is uncorrelated
with the contents of the ratings.
This would require very little extra server load, and minimal rewriting
of the ratings code, and could be made more scalable very quickly, by --
for example -- running multiple logging processes on multiple servers,
and sending them the ratings results at random.
I'm sure that a great many interested parties would be more than glad to
data-mine the results for us. As David Gerard has remarked, most of the
obvious objections to ratings schemes, such as trolling or rating
advocacy campaigns by article supporters or opponents, can be addressed
quite easily with quite simple statistical analysis techniques (for
example, see the analysis techniques used to mine the Seti@Home data
after it's been number-crunched).
Incidentally, a very similar technique could be used to analyze server
traffic; just syslog an entry (to a different log, of course) for one in
every 100, or one in every 1000, server hits.
-- Neil