This paper (first reference) is the result of a class project I was part of
almost two years ago for CSCI 5417 Information Retrieval Systems. It builds
on a class project I did in CSCI 5832 Natural Language Processing and which
I presented at Wikimania '07. The project was very late as we didn't send
the final paper in until the day before new years. This technical report was
never really announced that I recall so I thought it would be interesting to
look briefly at the results. The goal of this paper was to break articles
down into surface features and latent features and then use those to study
the rating system being used, predict article quality and rank results in a
search engine. We used the [[random forests]] classifier which allowed us to
analyze the contribution of each feature to performance by looking directly
at the weights that were assigned. While the surface analysis was performed
on the whole english wikipedia, the latent analysis was performed on the
simple english wikipedia (it is more expensive to compute). = Surface
features = * Readability measures are the single best predictor of quality
that I have found, as defined by the Wikipedia Editorial Team (WET). The
[[Automated Readability Index]], [[Gunning Fog Index]] and [[Flesch-Kincaid
Grade Level]] were the strongest predictors, followed by length of article
html, number of paragraphs, [[Flesh Reading Ease]], [[Smog Grading]], number
of internal links, [[Laesbarhedsindex Readability Formula]], number of words
and number of references. Weakly predictive were number of to be's, number
of sentences, [[Coleman-Liau Index]], number of templates, PageRank, number
of external links, number of relative links. Not predictive (overall - see
the end of section 2 for the per-rating score breakdown): Number of h2 or
h3's, number of conjunctions, number of images*, average word length, number
of h4's, number of prepositions, number of pronouns, number of interlanguage
links, average syllables per word, number of nominalizations, article age
(based on page id), proportion of questions, average sentence length. :*
Number of images was actually by far the single strongest predictor of any
class, but only for Featured articles. Because it was so good at picking out
featured articles and somewhat good at picking out A and G articles the
classifier was confused in so many cases that the overall contribution of
this feature to classification performance is zero. :* Number of external
links is strongly predictive of Featured articles. :* The B class is highly
distinctive. It has a strong "signature," with high predictive value
assigned to many features. The Featured class is also very distinctive. F, B
and S (Stop/Stub) contain the most information.
:* A is the least distinct class, not being very different from F or G. =
Latent features = The algorithm used for latent analysis, which is an
analysis of the occurence of words in every document with respect to the
link structure of the encyclopedia ("concepts"), is [[Latent Dirichlet
Allocation]]. This part of the analysis was done by CS PhD student Praful
Mangalath. An example of what can be done with the result of this analysis
is that you provide a word (a search query) such as "hippie". You can then
look at the weight of every article for the word hippie. You can pick the
article with the largest weight, and then look at its link network. You can
pick out the articles that this article links to and/or which link to this
article that are also weighted strongly for the word hippie, while also
contributing maximally to this articles "hippieness". We tried this query in
our system (LDA), Google (site:en.wikipedia.org hippie), and the Simple
English Wikipedia's Lucene search engine. The breakdown of articles occuring
in the top ten search results for this word for those engines is: * LDA
only: [[Acid rock]], [[Aldeburgh Festival]], [[Anne Murray]], [[Carl
Radle]], [[Harry Nilsson]], [[Jack Kerouac]], [[Phil Spector]], [[Plastic
Ono Band]], [[Rock and Roll]], [[Salvador Allende]], [[Smothers brothers]],
[[Stanley Kubrick]]. * Google only: [[Glam Rock]], [[South Park]]. * Simple
only: [[African Americans]], [[Charles Manson]], [[Counterculture]], [[Drug
use]], [[Flower Power]], [[Nuclear weapons]], [[Phish]], [[Sexual
liberation]], [[Summer of Love]] * LDA & Google & Simple: [[Hippie]],
[[Human Be-in]], [[Students for a democratic society]], [[Woodstock
festival]] * LDA & Google: [[Psychedelic Pop]] * Google & Simple: [[Lysergic
acid diethylamide]], [[Summer of Love]] ( See the paper for the articles
produced for the keywords philosophy and economics ) = Discussion /
Conclusion = * The results of the latent analysis are totally up to your
perception. But what is interesting is that the LDA features predict the WET
ratings of quality just as well as the surface level features. Both feature
sets (surface and latent) both pull out all almost of the information that
the rating system bears. * The rating system devised by the WET is not
distinctive. You can best tell the difference between, grouped together,
Featured, A and Good articles vs B articles. Featured, A and Good articles
are also quite distinctive (Figure 1). Note that in this study we didn't
look at Start's and Stubs, but in earlier paper we did. :* This is
interesting when compared to this recent entry on the YouTube blog. "Five
Stars Dominate Ratings"
http://youtube-global.blogspot.com/2009/09/five-stars-dominate-ratings.html…
I think a sane, well researched (with actual subjects) rating system
is
well within the purview of the Usability Initiative. Helping people find and
create good content is what Wikipedia is all about. Having a solid rating
system allows you to reorganized the user interface, the Wikipedia
namespace, and the main namespace around good content and bad content as
needed. If you don't have a solid, information bearing rating system you
don't know what good content really is (really bad content is easy to spot).
:* My Wikimania talk was all about gathering data from people about articles
and using that to train machines to automatically pick out good content. You
ask people questions along dimensions that make sense to people, and give
the machine access to other surface features (such as a statistical measure
of readability, or length) and latent features (such as can be derived from
document word occurence and encyclopedia link structure). I referenced page
262 of Zen and the Art of Motorcycle Maintenance to give an example of the
kind of qualitative features I would ask people. It really depends on what
features end up bearing information, to be tested in "the lab". Each word is
an example dimension of quality: We have "*unity, vividness, authority,
economy, sensitivity, clarity, emphasis, flow, suspense, brilliance,
precision, proportion, depth and so on.*" You then use surface and latent
features to predict these values for all articles. You can also say, when a
person rates this article as high on the x scale, they also mean that it has
has this much of these surface and these latent features.
= References =
- DeHoust, C., Mangalath, P., Mingus., B. (2008). *Improving search in
Wikipedia through quality and concept discovery*. Technical Report.
PDF<http://grey.colorado.edu/mediawiki/sites/mingus/images/6/68/DeHoustMangalat…>
- Rassbach, L., Mingus., B, Blackford, T. (2007). *Exploring the
feasibility of automatically rating online article quality*. Technical
Report. PDF<http://grey.colorado.edu/mediawiki/sites/mingus/images/d/d3/RassbachPincock…>
Hoi,
I have asked and received permission to forward to you all this most
excellent bit of news.
The linguist list, is a most excellent resource for people interested in the
field of linguistics. As I mentioned some time ago they have had a funding
drive and in that funding drive they asked for a certain amount of money in
a given amount of days and they would then have a project on Wikipedia to
learn what needs doing to get better coverage for the field of linguistics.
What you will read in this mail that the total community of linguists are
asked to cooperate. I am really thrilled as it will also get us more
linguists interested in what we do. My hope is that a fraction will be
interested in the languages that they care for and help it become more
relevant. As a member of the "language prevention committee", I love to get
more knowledgeable people involved in our smaller projects. If it means that
we get more requests for more projects we will really feel embarrassed with
all the new projects we will have to approve because of the quality of the
Incubator content and the quality of the linguistic arguments why we should
approve yet another language :)
NB Is this not a really clever way of raising money; give us this much in
this time frame and we will then do this as a bonus...
Thanks,
GerardM
---------- Forwarded message ----------
From: LINGUIST Network <linguist(a)linguistlist.org>
Date: Jun 18, 2007 6:53 PM
Subject: 18.1831, All: Call for Participation: Wikipedia Volunteers
To: LINGUIST(a)listserv.linguistlist.org
LINGUIST List: Vol-18-1831. Mon Jun 18 2007. ISSN: 1068 - 4875.
Subject: 18.1831, All: Call for Participation: Wikipedia Volunteers
Moderators: Anthony Aristar, Eastern Michigan U <aristar(a)linguistlist.org>
Helen Aristar-Dry, Eastern Michigan U <hdry(a)linguistlist.org>
Reviews: Laura Welcher, Rosetta Project
<reviews(a)linguistlist.org>
Homepage: http://linguistlist.org/
The LINGUIST List is funded by Eastern Michigan University,
and donations from subscribers and publishers.
Editor for this issue: Ann Sawyer <sawyer(a)linguistlist.org>
================================================================
To post to LINGUIST, use our convenient web form at
http://linguistlist.org/LL/posttolinguist.html
===========================Directory==============================
1)
Date: 18-Jun-2007
From: Hannah Morales < hannah(a)linguistlist.org >
Subject: Wikipedia Volunteers
-------------------------Message 1 ----------------------------------
Date: Mon, 18 Jun 2007 12:49:35
From: Hannah Morales < hannah(a)linguistlist.org >
Subject: Wikipedia Volunteers
Dear subscribers,
As you may recall, one of our Fund Drive 2007 campaigns was called the
"Wikipedia Update Vote." We asked our viewers to consider earmarking their
donations to organize an update project on linguistics entries in the
English-language Wikipedia. You can find more background information on this
at:
http://linguistlist.org/donation/fund-drive2007/wikipedia/index.cfm.
The speed with which we met our goal, thanks to the interest and generosity
of
our readers, was a sure sign that the linguistics community was enthusiastic
about the idea. Now that summer is upon us, and some of you may have a bit
more
leisure time, we are hoping that you will be able to help us get started on
the
Wikipedia project. The LINGUIST List's role in this project is a purely
organizational one. We will:
*Help, with your input, to identify major gaps in the Wikipedia materials or
pages that need improvement;
*Compile a list of linguistics pages that Wikipedia editors have identified
as
"in need of attention from an expert on the subject" or " does not cite any
references or sources," etc;
*Send out periodical calls for volunteer contributors on specific topics or
articles;
*Provide simple instructions on how to upload your entries into Wikipedia;
*Keep track of our project Wikipedians;
*Keep track of revisions and new entries;
*Work with Wikimedia Foundation to publicize the linguistics community's
efforts.
We hope you are as enthusiastic about this effort as we are. Just to help us
all
get started looking at Wikipedia more critically, and to easily identify an
area
needing improvement, we suggest that you take a look at the List of
Linguists
page at:
http://en.wikipedia.org/wiki/List_of_linguists. M
Many people are not listed there; others need to have more facts and
information
added. If you would like to participate in this exciting update effort,
please
respond by sending an email to LINGUIST Editor Hannah Morales at
hannah(a)linguistlist.org, suggesting what your role might be or which
linguistics
entries you feel should be updated or added. Some linguists who saw our
campaign
on the Internet have already written us with specific suggestions, which we
will
share with you soon.
This update project will take major time and effort on all our parts. The
end
result will be a much richer internet resource of information on the breadth
and
depth of the field of linguistics. Our efforts should also stimulate
prospective
students to consider studying linguistics and to educate a wider public on
what
we do. Please consider participating.
Sincerely,
Hannah Morales
Editor, Wikipedia Update Project
Linguistic Field(s): Not Applicable
-----------------------------------------------------------
LINGUIST List: Vol-18-1831
Tim Starling wrote:
> According to ru.wp Arbcom member DR, the danger to Wikipedia was
> overstated, and the text of the proposed law was misrepresented.
I think that the interpretation to the bill given by DR is incorrect. In fact the proposed bill was not only about child pornography and extremism, but also about drugs and, about “information, prompting children to commit actions, making threat to their life and health”. That was a very loose clause, that could ban virtually anything. After the blackout this clause was removed from the bill and it is a clear achievement of the strike. On the other hand the final version of the bill contains another clause, that is even more hazardous to us. It is about “information of methods of producing and use of narcotic substances, … of methods and places of cultivation of narcotic plants”. We do have information of drug synthesis on Wikipedia, ways of its use (e.g. marijuana) and we do have thorough instructions of marijuana cultivation on wikibooks. That is why our achievements are ambiguous. On the one hand we have a removal of a loose clause about information harmful to children, but on the otherwe now have another clause that is even more dangerous. That is why we are still trying to do what we can via our contacts within the authorities to revise the passed bill.
But that is not all. The most important issue is extremism. According to the bill, the materials, that are banned for distribution in Russia should be included to the register of banned information on the ground of the court decision, banning the distribution of that information in Russia. We already have such court decisions and a list of extremist materials, distribution of which is prohibited in Russia. That list contains some really nasty materials, as e.g. nazi propaganda, but also Islamic texts (including those of famous non-terrorist Islamic authors e.g. Said Nursî), Saentologist, Jehova’s witnesses , Falun Gong, letters and materials of opposition in Russia, works of contemporary art, etc.
We *do have* banned extremist materials in Wikipedia. E.g. this image:
http://ru.wikipedia.org/wiki/Файл:Александр_Савко_Путешествия_Микки_Мауса_п… искусства.jpeg
is considered extremist and is banned for distribution in Russia. (Hopefully it was uploaded two years before it was regulated as banned by the court).
This letter in wikisource is also considered extremist:
http://ru.wikisource.org/wiki/Всем,_кто_сочувствует_жертвам_бесланского_тер…!
This is enough for banning the IPs of Wikimedia projects in Russia. And I am really afraid of this.
I guess DR is aware of discussion on this list, but anyway I will inform him of it. Maybe he has something to add.
> According to Levg in his Arbcom application, again via Google
> Translate, "It should be noted that there are no objective reasons for
> such a 'sprint survey' did not exist, to discuss the bill on second
> reading has been known since at least last Friday."
That is our fault that we could not manage to get the information in time. The first hearing was on Friday, but the community and myself got to know about the problem only on Monday, 9th. What for me personally I haven’t read the news on the weekend (yes, it is bad, that I relaxed on the weekend and haven’t read the news), and I failed to get to know about the problem in time. I guess it is also true for others. If we start to organize on Friday, the result would be better. It is a fault, but anyway it was not a deliberate fault, as nobody has informed the community earlier.
*Hi everyone,
It's been a bit since I last emailed this list (or any list, for that
matter!)... you may remember me, I worked at the Foundation last year in
the Community Department, working with Philippe on any number of issues,
as well as with the OTRS team. I've come back to work on a short term
project with the Foundation, and I have to say it's great to be back!
(and a great break from my Master's thesis!)
We're getting ready to run the next version of the Editor Survey, for
August 2012. This will be the third incarnation we've run since 2011.
As with the prior incarnations of the survey, we'll be looking at a
variety of topics, this time with the goal of not only understanding
your needs and pressing issues while interacting with fellow editors,
but also focusing on editors' satisfaction with the work of the Foundation.
The last time we ran an editor survey, it was completed by over 6,000
respondents. When you break that down, it means that each minute of
time demanded by the survey corresponds to 100 hours of Wikipedians'
time. We want to make sure that this time is spent wisely, ensuring
that the questions we have are worded clearly, don't cause confusion,
and will generate meaningful answers. So we'd like to ask you to take a
look at the survey, and give us feedback on the questions. You can find
them here:
https://meta.wikimedia.org/wiki/Research:Wikipedia_Editor_Survey_August_201…
<https://meta.wikimedia.org/w/index.php?title=Research:Wikipedia_Editor_Surv…>
... and please leave your feedback on the talk page there so we can keep
the discussion in one place :)
You can find out more information about the survey here:
https://meta.wikimedia.org/wiki/Research:Wikipedia_Editor_Survey_August_2012
Also, we are planning an IRC Office Hour on the survey, this **Tuesday,
July 31 at 1700 UTC.** (See
https://meta.wikimedia.org/wiki/IRC_office_hoursfor general information
about IRC Office hours)
I know there has been some discussion about offering Office Hours in a
broader range of times, and I know this time may not be the greatest for
some... but this was the best time we could find currently.
Thanks everyone!!
-Christine
Wikimedia Foundation*
Hi.
How is the fund-raising goal determined each year?
Is there a fund-raising goal set for the upcoming fund-raiser (the one
beginning in November 2012)?
Is there a guideline or policy regarding what happens once the fund-raising
goal is met?
MZMcBride
On 7/28/12 5:58 AM, Tilman Bayer wrote:
> Hi all,
>
> the Wikimedia Foundation's 2012-13 Annual Plan has just been published at
>
> https://wikimediafoundation.org/wiki/File:2012-13_Wikimedia_Foundation_Plan…
>
> accompanied by a Q&A:
>
> https://wikimediafoundation.org/wiki/2012-2013_Annual_Plan_Questions_and_An…
>
> The plan was approved by the Board of Trustees at its meeting in
> Washington, DC, at Wikimania, and previously outlined to the
> Foundation staff and interested community members at the monthly staff
> meeting on July 5, 2012. We were planning to publish the video
> recording of that meeting at this point, but encountered technical
> difficulties; the video will hopefully become available soon.
>
Slide 8 : "How are we doing against the 2012 targets"
I was stopped by
"The Global Education Program is now the largest-ever systematic effort
of the Wikimedia mouvement to boost high quality content creation, with
a projected addition of 19 million characters to Wikipedia through
student assignements 2011-2012"
OF COURSE, we all know that WMF needs to glorify what it is actually
initiating/in charge of. And that's fair enough.
But seriously... I would feel fine with us trying to claim that the GEP
is the largest system effort to INCREASE the number of articles. It is
probably true.
But we all know that the result is... so and so. Possibly good content,
but also lot's of crap being reverted and deleted afterwards. Claiming
it is the largest effort to boost high quality content is not only
disingenous... but I actually find it counter productive and a tiny bit
offensive toward the actual community.
High quality content simply does NOT come from newbie students.
Florence
Hi,
In the 2012-13 WMF plan document I saw an interesting thing:
"We’ve hosted key community stakeholders such as English Wikipedia’s
ArbCom and Portuguese Wikipedia’s top contributors, in an effort to
better understand and respond to issues they're facing." (page 41).
I was very happy to read this. In general, I hope that such focused
meetings will be held with more language communities. I don't think
that I need to explain why :)
I don't know how did the meeting with the Portuguese Wikipedians go; I
suppose that it was good. I don't remember that I read anything about
it in blogs or mailing lists, but I may have missed it. Maybe what I'm
about to write is known already, but I'll say it anyway.
An important thing in such meetings is to have a community member who
contributes to the Wikipedia in that language AND to the English
Wikipedia. This is needed because the Foundation people are probably
familiar with policies, customs and jargon in the English Wikipedia.
Even simple terms, like "Village Pump", are not necessarily familiar
to people who primarily edit in other languages; not all Wikipedias
have ArbComs; not all Wikipedias prohibit voting; etc. Such a person
will be able to "translate" between the English Wikipedia terms and
the local Wikipedia terms. Without such a person misunderstandings
will definitely happen, even if everybody knows the English language
well.
That's it, hope it helps.
--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore
Hi all,
the Wikimedia Foundation's 2012-13 Annual Plan has just been published at
https://wikimediafoundation.org/wiki/File:2012-13_Wikimedia_Foundation_Plan…
accompanied by a Q&A:
https://wikimediafoundation.org/wiki/2012-2013_Annual_Plan_Questions_and_An…
The plan was approved by the Board of Trustees at its meeting in
Washington, DC, at Wikimania, and previously outlined to the
Foundation staff and interested community members at the monthly staff
meeting on July 5, 2012. We were planning to publish the video
recording of that meeting at this point, but encountered technical
difficulties; the video will hopefully become available soon.
--
Tilman Bayer
Senior Operations Analyst (Movement Communications)
Wikimedia Foundation
IRC (Freenode): HaeB
_______________________________________________
Please note: all replies sent to this mailing list will be immediately directed to Wikimedia-l, the public mailing list of the Wikimedia community. For more information about Wikimedia-l:
https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
_______________________________________________
WikimediaAnnounce-l mailing list
WikimediaAnnounce-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimediaannounce-l
Hi,
The Olympic games are beginning soon. Apparently, ticket holders
cannot use photo equipment longer than 30cm and cannot use the photos
and videos for commercial purposes without accreditation.
Practically everything that happens at the Olympics is notable and
should be on Wikipedia, Commons, etc. Does anybody know whether there
are professional accredited photographers who are Wikimedia-friendly
and plan to upload their photos? If there aren't any, does anybody
know whether a Wikipedian can obtain such accreditation?
This doesn't concern me directly, but there are many, many people who
write Wikipedia articles about sports in all languages and it may be
interesting to them. Also, it may be a frequent issue in sports and
I'm just not aware of it because I rarely follow sports.
Sources for the restrictions:
* http://www.tickets.london2012.com/purchaseterms.html
* PDF: http://j.mp/london2012prohibited
--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore