[Wikipedia-l] Measuring progress

[Michael R. Irwin]

Axel Boldt wrote:
> 
> > our progress has not stalled (right?)
> 
> If you measure progress based on article numbers, active contributors,
> edits per day, software quality etc., it certainly hasn't.
> 
> I would argue however that the most relevant measure is average
> article quality. And Larry seems to think that on this front, we're
> not doing too well:
> 
> > But, looking at Wikipedia's contents now and comparing it to what I
> > recall from times past, I do have to say that I'm worried. I don't
> > think that in terms of quality, overall, it's getting that much
> > better.
> 
> While it is clear that pretty much every individual article improves
> over time, it is still very possible that the average article quality
> is stagnating, or even declining.
> 
> If I had an afternoon to burn, I'd hit the "Random link" button a
> hundred times and would rate the articles on a scale of 0 - 10. Then I
> would rate 100 article versions from three months ago (skip if the
> article didn't exist then). Then compare the averages. Ideally, this
> would be done in a "blind" manner, so that I didn't know whether an
> article version is current or old.

How would you (and the mailing list) rate the following
in terms of potential usefulness:

A quality inspection page which would take the following
data:  Number of articles to inspect, comparison period or
dates, logged in account, other?

present the number of articles specified (using random dates
or revisions if the dates and period are not specified) sequentially
for qualitative assessment on a standard predetermined scale,

add the response into an aggregate data set for plotting
or other evaulation.

Would something similar to the above allow a rough 
community consensus on our quality trends to emerge?

I suppose one could also keep results of the individual 
data runs to personally calibrate your personal assessment
against the results of the community.

If this were compiled properly it might also show us some 
trends in community perception of quality related to educational
background, sex, etc. if any of that is perceived as useful.

Personally I suspect we currently have a moving target
in terms of perception of quality.   Perhaps we would
need a data field in the above for length of participation,
number of edits in some period, number of contributions
recorded, etc.  to properly guage reactions of newcomers
or occasional readers compared to regular contributors.

Would we need to calibrate the data against specific
articles or categories?   We may need an experienced
statitician to weigh in here if we get serious about
anything like this. 

regards,
Mike Irwin