Dear All,
we have three new techreps available:
- Robust Content-Driven
Reputation<http://www.soe.ucsc.edu/%7Eluca/papers/08/ucsc-soe-08-09.html>shows
that the content-driven reputation we proposed in a WWW 2007 paper can
be made robust to Sybil ("sock-puppet") and other coordinated attacks. In
WWW 2007, we proposed "content-driven reputation" for Wikipedia authors,
where authors gain reputation if their contributions are preserved, and lose
reputation if their contributions are quickly undone. The original
algorithms were very prone to attacks; we show here that they can be made
resistant.
- Assigning Trust to Wikipedia
Content<http://www.soe.ucsc.edu/%7Eluca/papers/08/ucsc-soe-08-07.html>proposes
computing the trust of Wikipedia text on the basis of the
reputation of the author, and the reputation of the people who revised the
text. We display text trust by coloring text background. Many of you have
seen the on-line demo for the English Wikipedia, at
http://trust.cse.ucsc.edu/ . This is an improved version of a November
2007 techrep on the same topic. In this improved techrep, we show how the
trust system can be made resistant to attacks.
- Measuring Author Contributions to the
Wikipedia<http://www.soe.ucsc.edu/%7Eluca/papers/08/ucsc-soe-08-08.html>defines
and compares various ways for measuring the contribution of
individual authors to the Wikipedia. We have our own favorite; read more to
find out :-)
In these months, we have been busy working at
WikiTrust<http://trust.cse.ucsc.edu/>,
an open-source tool for assigning reputation to wiki authors and trust to
wiki content. We already have a batch (or "off-line") system, which can
compute reputation and trust based on wiki dumps, such as the Wikipedia
dumps made available by the Wikimedia Foundation. We are developing an
"on-line" system, which can assign reputation and trust in real-time, as
edits are made. One of our chief concerns in developing an on-line system
was to ensure that it was robust to attack, and we believe we have made
progress in this direction, as reported in the above techreps. We are now
proceeding with the implementation; my guess is that we will have a
prototype in a month or so.
By the way, the "batch" part of WikiTrust <http://trust.cse.ucsc.edu/> can
be easily adapted to carry out various analysis tasks. Basically, it walks
over all revisions of every page of a wiki, and it contains an efficient
text analysis engine that tells you precisely how text was changed between
versions. So, it is easy to use WikiTrust as a platform to write analysis
algorithms for wikis: you don't have to worry about the boring tasks of
reading and parsing markup language, and computing text diffs in a
reasonable way; you can concentrate on the details of the specific analysis
you want to do. It is all open source, and we welcome developers or people
interested in it.
All the best,
Luca (with Ian, Bo, and the other wikitrusters).
Hiho,
as most of you will have heard, flagged revisions were turned on on
de.wikipedia.org. You can follow progress on
http://tools.wikimedia.de/~aka/cgi-bin/reviewcnt.cgi. Now, it would be
important to measure success of flagged revisions in some way. The
following metrics come to my mind:
-Number of articles with a sighted revision (not very useful though,
but measures acceptance among editors in a way.)
-Number of articles that have a sighted revision but where the current
version is not sighted
-Time needed to sight revisions (max and mean of time until a revision
by a noneditor is sighted. The mean is very difficult to get, but
could be computed by using the mean of the pages in
Spezial:OldReviewedPages)
-Number of editors, meaning users who have the right to sight edits
(again, acceptance but also to see if we hinder people in editing more
than we should)
Do you have more ideas for metrics and how to measure them?
Best,
Philipp
Phil,
Thanks for your note. I have not made any suggestion or concurrences yet, but suggest you edit your note:
second paragraph: change"sighted" to "cited." "Sighted" is what happens to UFO's. "Cited" is what happens to
reference articles.
Otherwise, cool.
BudgieBirdChan0211
----- Original Message ----
From: P. Birken <pbirken(a)gmail.com>
To: Wikimedia Quality Discussions <wikiquality-l(a)lists.wikimedia.org>
Sent: Wednesday, May 14, 2008 6:13:17 AM
Subject: [Wikiquality-l] Metrics for Testing of Flagged Revs
Hiho,
as most of you will have heard, flagged revisions were turned on on
de.wikipedia.org. You can follow progress on
http://tools.wikimedia.de/~aka/cgi-bin/reviewcnt.cgi. Now, it would be
important to measure success of flagged revisions in some way. The
following metrics come to my mind:
-Number of articles with a sighted revision (not very useful though,
but measures acceptance among editors in a way.)
-Number of articles that have a sighted revision but where the current
version is not sighted
-Time needed to sight revisions (max and mean of time until a revision
by a noneditor is sighted. The mean is very difficult to get, but
could be computed by using the mean of the pages in
Spezial:OldReviewedPages)
-Number of editors, meaning users who have the right to sight edits
(again, acceptance but also to see if we hinder people in editing more
than we should)
Do you have more ideas for metrics and how to measure them?
Best,
Philipp
_______________________________________________
Wikiquality-l mailing list
Wikiquality-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikiquality-l
----- Forwarded Message ----
From: Norman Kelley <pinot_chan0730(a)yahoo.com>
To: Wikimedia Quality Discussions <wikiquality-l(a)lists.wikimedia.org>
Sent: Wednesday, May 14, 2008 12:34:16 PM
Subject: Re: [Wikiquality-l] "sighted", and another metric
Is not this suggestion a variation on rule by consensus? What about objective standards?
Whatever we agree on (consensus) it must be auditable.
Norman
----- Original Message ----
From: David Gerard <dgerard(a)gmail.com>
To: Dan Kindsvater <dank55(a)gmail.com>; Wikimedia Quality Discussions <wikiquality-l(a)lists.wikimedia.org>
Sent: Wednesday, May 14, 2008 11:22:36 AM
Subject: Re: [Wikiquality-l] "sighted", and another metric
2008/5/14 Dan Kindsvater <dank55(a)gmail.com>:
> The most important metric to me and many editors is how many quality
> articles we're producing. A major reason for Flagged Versions is that
> various content experts have told us that this is one thing they need if we
> want their help. So, one important metric I'd like to suggest is to survey
> and count the number of new editors who say that the new feature is a major
> reason they decided to start participating.
Also: note that Wikinews (en:wn at least) really wants sighted/stable
versions displayed by default so that they will make it into
aggregators, specifically Google News.
- d.
_______________________________________________
Wikiquality-l mailing list
Wikiquality-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikiquality-l
_______________________________________________
Wikiquality-l mailing list
Wikiquality-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikiquality-l
Is not this suggestion a variation on rule by consensus? What about objective standards?
Whatever we agree on (consensus) it must be auditable.
Norman
----- Original Message ----
From: David Gerard <dgerard(a)gmail.com>
To: Dan Kindsvater <dank55(a)gmail.com>; Wikimedia Quality Discussions <wikiquality-l(a)lists.wikimedia.org>
Sent: Wednesday, May 14, 2008 11:22:36 AM
Subject: Re: [Wikiquality-l] "sighted", and another metric
2008/5/14 Dan Kindsvater <dank55(a)gmail.com>:
> The most important metric to me and many editors is how many quality
> articles we're producing. A major reason for Flagged Versions is that
> various content experts have told us that this is one thing they need if we
> want their help. So, one important metric I'd like to suggest is to survey
> and count the number of new editors who say that the new feature is a major
> reason they decided to start participating.
Also: note that Wikinews (en:wn at least) really wants sighted/stable
versions displayed by default so that they will make it into
aggregators, specifically Google News.
- d.
_______________________________________________
Wikiquality-l mailing list
Wikiquality-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikiquality-l
>change"sighted" to "cited."
"Sighted" is what was meant, it's a direct translation of the German
"gesichtet". It does sound slightly awkward in English, and we've been
using the generic "flagged" to describe articles that are "quality",
"sighted", etc. It's probably time to pick a new word.
The most important metric to me and many editors is how many quality
articles we're producing. A major reason for Flagged Versions is that
various content experts have told us that this is one thing they need if we
want their help. So, one important metric I'd like to suggest is to survey
and count the number of new editors who say that the new feature is a major
reason they decided to start participating.
Dan, Dank55 on en.wp
----- Original Message ----
From: Norman Kelley <pinot_chan0730(a)yahoo.com>
To: Wikimedia Quality Discussions <wikiquality-l(a)lists.wikimedia.org>
Sent: Wednesday, May 14, 2008 9:13:22 AM
Subject: Re: [Wikiquality-l] Metrics for Testing of Flagged Revs
Phil,
Thanks for your note. I have not made any suggestion or concurrences yet, but suggest you edit your note:
second paragraph: change"sighted" to "cited." "Sighted" is what happens to UFO's. "Cited" is what happens to
reference articles.
Otherwise, cool.
BudgieBirdChan0211
----- Original Message ----
From: P. Birken <pbirken(a)gmail.com>
To: Wikimedia Quality Discussions <wikiquality-l(a)lists.wikimedia.org>
Sent: Wednesday, May 14, 2008 6:13:17 AM
Subject: [Wikiquality-l] Metrics for Testing of Flagged Revs
Hiho,
as most of you will have heard, flagged revisions were turned on on
de.wikipedia.org. You can follow progress on
http://tools.wikimedia.de/~aka/cgi-bin/reviewcnt.cgi. Now, it would be
important to measure success of flagged revisions in some way. The
following metrics come to my mind:
-Number of articles with a sighted revision (not very useful though,
but measures acceptance among editors in a way.)
-Number of articles that have a sighted revision but where the current
version is not sighted
-Time needed to sight revisions (max and mean of time until a revision
by a noneditor is sighted. The mean is very difficult to get, but
could be computed by using the mean of the pages in
Spezial:OldReviewedPages)
-Number of editors, meaning users who have the right to sight edits
(again, acceptance but also to see if we hinder people in editing more
than we should)
Do you have more ideas for metrics and how to measure them?
Best,
Philipp
_______________________________________________
Wikiquality-l mailing list
Wikiquality-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikiquality-l
Kindly clarify this topic. I think the original request was inadvertently deleted.
Thank you.
____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ