Good question. We used the current revisions (at snapshot time). You are right, there may be some tagged articles were the flaw has already been fixed. We used the current revisions because this is the first competition on quality flaw prediction and we want to keep the hurdles for the participants as low as possible. The computation of particular article features for a certain revision is relative complex, consider for instance the number of incoming links. By contrast, it is relative easy to compute these features on a fixed snapshot, for instance by utilizing the respective database dumps.

Best regards,
Maik

On 23.03.2012 14:06, emijrp wrote:
Just a simple question... have you used the last revision for every tagged article, or the revision when the tag was added? Sometimes the issue is corrected but the tag is not removed until days, weeks, months... later.

2012/3/21 Maik Anderka <maik.anderka@uni-weimar.de>
-------------------------------------------------------------------------------
Call for Participation - Competition on Quality Flaw Prediction in Wikipedia
-------------------------------------------------------------------------------

The competition is part of the PAN 2012 Lab,
held in conjunction with the CLEF'12 conference in Roma, Italy.

In the previous PAN Labs, we have addressed quality issues in Wikipedia in the
form of vandalism. However, the majority of quality flaws is not caused due to
malicious intentions but stem from edits by inexperienced authors; examples
include poor writing style, unreferenced statements, or missing neutrality.
This year, we generalize the vandalism detection task and focus on the
prediction of particular quality flaws in Wikipedia articles.

We invite researchers and practitioners from all fields to participate.

Find out about all the details at http://pan.webis.de.

-------------------------------------------------------------------------------
Important Dates:

now open          Registration
Mar 16, 2012      Training data release
May 18, 2012      Test data release
Jun 01, 2012      Run submission
Jun 22, 2012      Abstract submission
Jul 06, 2012      Notification
Aug 10, 2012      Notebook submission
Sep 17-20, 2012   Conference

-------------------------------------------------------------------------------
Task Committee:

Maik Anderka and Benno Stein
Webis @ Bauhaus-Universität Weimar, Germany


_______________________________________________
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l


-- 
Maik Anderka
Web Technology and Information Systems
Faculty of Media, Computer Science and Media 
Bauhaus-Universität Weimar
http://www.webis.de/