http://ksh.wikipedia.org/wiki/Varsseveld It's not Ripuarian (ksh), but Nedersaksisch, the text is taken directly from nds-nl. Ziko

2008/6/27 Andre Engels andreengels@gmail.com:

On Fri, Jun 27, 2008 at 3:49 PM, Ziko van Dijk zvandijk@googlemail.com wrote:

...

A pseudo article is e.g. http://pdc.wikipedia.org/wiki/Bikini http://co.wikipedia.org/wiki/191 http://ksh.wikipedia.org/wiki/Varsseveld http://pl.wikipedia.org/wiki/Tandil http://vo.wikipedia.org/wiki/Poplar_Bluff

Ok, I understand numbers 2, 4 and 5 in your list. Number 1 is presumably included for being extremely stubby, but what's the issue with the ksh: page? Only thing I notice is that the text part hasn't got any internal links. But to consider something like that a 'non article' like the co: and pl: examples seems harsh in the extreme.

-- Andre Engels, andreengels@gmail.com ICQ: 6260644 -- Skype: a_engels

---

foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

Alas, judging a language edition by Wikimedia Statistics does not work.

Indonesian, Asturian and Volapük WPs have the same "depth" (8), but id.WP is a very good WP. How comes? There not so many edits per article in id.WP, because it has translated a lot from English. A legitimate way to create (good) articles, but it does not need a lot of edits.

Bot activity: Indeed, "bot bumps" can often easily be detected in stats tables. Especially the small Wikipedias (I suppose) show (relatively) many bot activities due to interwiki linking. On the other hand, pseudo articles can be created by hand (let a script create it outside WP and then insert it "manually").

Ziko

2008/6/27 Harel Cain harel.cain@gmail.com:

The depth criterion available here: http://meta.wikimedia.org/wiki/List_of_wikipedias is a good starting point. I quote: "The "Depth" column ((Edits/Articles) × (Non-Articles/Articles) × (Stub-ratio)) is a rough indicator of a Wikipedia's quality, showing how frequently its articles are updated."

Note that indeed Volapuek, Polish, Ripuarian and others have very low depth ranking.

Harel

On Fri, Jun 27, 2008 at 4:49 PM, Ziko van Dijk zvandijk@googlemail.com wrote:

...

It would be good if the community found a different way to compare or to measure it's successes.

-- Quidquid latine dictum sit, altum viditur.

---

foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

2008/6/27 Ziko van Dijk zvandijk@googlemail.com:

Among the Big Wikipedias, the pl.WP has one of the lowest quota of real articles:

   Artikel (off.)  realt. Art.  Artikel W (Quot.)

EN 1400000 1344000 0,96 DE 696000 668160 0,96 FR 613000 514920 0,84 JA 466000 466000 1 IT 408000 301920 0,74 PL 467000 298880 0,64 ES 326000 293400 0,9 NL 404000 274720 0,68 SV 272000 217600 0,8 PT 338000 209560 0,62 RU 233000 195720 0,84 ZH 164000 144320 0,88 (most numbers from jan. 2008, en, de and pt older; estimations should be rounded, in fact)

Can you explain how this evalution been done? How do you distinguish between "real" and other articles? Especially I don't believe in statiscts shown for en Wikipedia. I have a feeing that there is much more bot created articles in en Wikipedia than your statistcs show.

About a year ago I wanted to evaluate the number of bot created articles created in Polish Wikipedia, and then evaluate how many of them were expanded by humans. Unfortunatelly it was impossible to perform as the bot owners do not keep records of its activity. Anyway we checked randomly what happened with bot-created articles about Polish villages and small towns, which was the very first bot produciton in our Wikikipedia. As I was strongly opposed several years ago to produce bot-created articles but failed to persuade my fellow wikipedians, I just wanted to prove that it was indeed bad idea. However, the study shown that around 70% of them were efectively expanded by humans. Villagers added quite a lot of useful stuff to these articles like histories of their villages, pictures of interesting buildings etc. Can you explain if these articles are treated "real" or "not real" in your statistics and why?

2008/6/28 Andre Engels andreengels@gmail.com:

On Sat, Jun 28, 2008 at 9:47 AM, Tomasz Ganicz polimerek@gmail.com wrote:

...

Can you explain how this evalution been done? How do you distinguish between "real" and other articles? Especially I don't believe in statiscts shown for en Wikipedia. I have a feeing that there is much more bot created articles in en Wikipedia than your statistcs show.

That is described in his first mail: He did 'random article' 50 times and used that as a sample.

Well it is not described - I mean there is no clear criteria of evaluation mentioned. Does he speak Japanese or Polish? Is it possible to recognize "real" and "unreal" articles without understanding them?

Compare:

http://he.wikipedia.org/wiki/%D7%9C%D7%95%D7%93%D7%96%27

Is it "real" or "unreal" article and why? I have a feeling that it is bot created, but I am no sure about it, as I don't speak Hebrew :-)

And what about this:

http://uk.wikipedia.org/wiki/%D0%A4%D1%96%D0%B3%D1%83%D0%BB%D1%81_%D1%96_%D0...

It is quite long, but I am almost sure that it is bot created and untouch by any human, because it contains only statistical data and sentences looking as if they were machine created. I don't speak Ukrainian well but understand it a little bit. But it is still just my feelings...

It is funny that this article is longer than similar in es-Wikipedia, although Spanish one was edited by humans for sure :-)

http://es.wikipedia.org/wiki/F%C3%ADgols_y_Ali%C3%B1%C3%A1

and moreover - if you check all Wikipedias which contain article about Fígols i Alinyà only Spanish one looks as edited by human (but it is just my feelings I can be wrong).

And this:

http://ta.wikipedia.org/wiki/%E0%AE%B5%E0%AE%BE%E0%AE%B0%E0%AF%8D%E0%AE%9A%E...

real or not real? I really don't know, probably bot-created :-)

I think if we would like to perform serios evaluation of "real" and "unreal" articles it should be based on clear, not based on "feelings" criteria, done on larger samples (at least 500 articles) and by people who understand what they are reading.

2008/6/28 Ziko van Dijk zvandijk@googlemail.com:

There is Google Translater, and the Interwikis help as well. That article of he.WP about Lodz I would count as a real article, because there is information more than in a data base (links to Holocaust related articles, something about 19th century, economy (textile)). Indeed, I would like to make a more scientific scheme and apply it to a larger sample, maybe there will establish a research group about. I believe that my method does give a reasonable picture; of course, whether my results say "50.000" real articles or "52.000" is not really a measurable difference.

Sorry about it, but it only shows that your results are not reliable, because it is based on your feelings and poor quality machine translations which could change in unpredictable way your feelings. I might be affraid that the results shown in your table is just a reflection of: a) the quality of machine translation performed by google - it is better for latin and germanic based languages (English, French, Italian, German, Dutch etc.) and much worse for slavic, arabic and East Asian languages. b)your own subconcious attitude toward various nations and Wikipedias - even if you are trying to evaluate them all fair

Google translate produces sometimes really funny results when translating from Polish to English. For example:

"Przyszłość partii przyszłością narodu" (Future of the party is the future of the nation) is translated to:

"The future of the future of the nation lot" :-)

Or:

"Byłbym spał, gdybym mógł." ( I would sleep, if only I could)

is translated to:

"I would be he lay, if I only could."

http://en.wikipedia.org/wiki/Machine_translation_software_usability#Trustwor...

http://www.nist.gov/speech/tests/mt/2006/doc/mt06eval_official_results.html

I think that a method to distunguish between "real" and "unreal" articles should be based on analysis of the history of article and formal "hard" criteria.

For example one can make a criteria that if there are at least 4 sentences writen by a human it is "real article".

2008/6/28 Ziko van Dijk zvandijk@googlemail.com:

I have discussed my study with many people (one had similar results), but no one was so aggressive, Tomasz.

...

b)your own subconcious attitude toward various nations and Wikipedias

? Is this an accusation?

No, I am just a scientist, so I have a tendency to be sceptical and have basic knowledge about typical mistakes of doing statistical research.Too small sample, no clear criteria of evaluating it, and you did not tested the experimental error or replication of your method, by comparing results from several experiments asking other people to use your meaning of what "real" article is.

50 articles sample tested by one person, who for sure have its own attitudes is not enough to say that this or another Wikipedia is better or worse. Everyone has its own attitudes towards one or another nation. It is very natural thing. And if there is no clear definition of what is "real" article and what is not, and to evaluate this it was used google machine translation (which according to NIST survey from 2006 is found to be OK in only around 49% cases) so I am quite sure that your results cannot be taken seriously. You could have stastical error at least around 15-20% (if not more), so the results 0,60 or 0,80 is in experimental error range.

Anyway it would be interesting to make better planned experiments to evaluate the quality of Wikipedia articles, but for sure it has to be done on larger sample, some sort of "hard" criteria or a group of at least 10 researchers speaking diffrent languages and having different cultural background when to use "soft, human based" criteria.

2008/6/29 Mark Williamson node.ue@gmail.com:

Second of all, Google consistently fared better than almost every other system, a surprising feat for a very recently developed system.

Not that recently - it's based on SYSTRAN.

- d.

Tomasz Ganicz wrote:

And if there is no clear definition of what is "real" article and what is not,

Apparently it was the 500k article event that caused Ziko to bring the topic up this time. He's frustrated (and so am I) that 500K articles is reported as an achievement, when it is indeed doubtful what quality these articles have. Still, I think he exaggerates the problem.

Earlier this year, when the topic came up on meta, it was because of which languages were featured as the top 10 on www.wikipedia.org, http://meta.wikimedia.org/wiki/Top_Ten_Wikipedias

Since then, the Russian Wikipedia has gained the 10th position and Swedish ("the one with all the stubs") is down to 11th, so there is one problem less to care about. During that discussion, I proposed to use the size of the compressed database dump (pages-articles.xml.bz2) as the official metric, since it both counts the total database size (one long article counts the same as two short ones) and it completely removes the impact of bot generated articles. The compressed size of the Volapük Wikipedia is very small, becase the same patterns appear in many of its numerous articles.

On the talk page, there is a table where this is shown, and you can sort by column by clicking the little boxes, http://meta.wikimedia.org/wiki/Talk:Top_Ten_Wikipedias#What_problem_do_we_wa...

I'd like to propose a quality metric: The difference in rank between the article count and the compressed database size.

The English Wikipedia is the biggest (rank 1), whether you count articles or compressed database size. So its quality is 0.

The Polish Wikipedia was the 4th by article count, but the 7th by compressed database size, for a quality of 4 - 7 = -3.

The Swedish Wikipedia was (when this table was compiled) the 10th biggest by article count, but the 12th biggest by compressed database size, so its quality is 10 - 12 = -2.

The Russian Wikipedia was the 11th by article count, but 9th by compressed database size, so its quality is +2. This doesn't mean the Russian Wikipedia is better than the English one, only that it is better than (two of) its peers of similar size.

The Volapük Wikipedia was the 15th by article count, but the worse than the 30th by compressed database size (the table is incomplete), so its quality is worse than -15.

2008/6/29 Stephen Bain stephen.bain@gmail.com:

On Sun, Jun 29, 2008 at 10:03 AM, Lars Aronsson lars@aronsson.se wrote:

...

I'd like to propose a quality metric: The difference in rank between the article count and the compressed database size.

I think this is a good metric, especially because it's a relative metric (since it's effectively comparing projects against their peers to see how mature they are).

Someone earlier was discussing article sizes, so I hacked up a script to graph the distribution of article sizes:

http://www.toolserver.org/~thebainer/articlesizes/

Yes, but size of avarage article can be easily icreased artifically by bot-creation of inofoboxes, navigation templates, long list of categories and interwiki etc.

Take a look for example on:

http://pl.wikipedia.org/wiki/Telmisartan

infobox is around 90% of its content. Blame Polish Wikipedia that it allows creating such articles, I would agree with you :-) The article was created by human not by bot.

Higher average size of articles in any Wikipedia can be easilly achieved just by creating bot-only articles with plenty of statstical data, huge infobox and several navigation templates. See example, I have already shown:

http://uk.wikipedia.org/wiki/%D0%A4%D1%96%D0%B3%D1%83%D0%BB%D1%81_%D1%96_%D0...

Would be nice to have a tool comparing the "real" size of articles, by which I mean counting size of free text only - without all templates and other non-text stuff.

Stephen Bain wrote:

Swedish Wikipedia is distributed in almost exactly the same way as English Wikipedia, with the difference being that its average size is less than half that of En's, at around 1900 bytes.

In Sweden we have this problem that people in our capital are so arrogant towards the rest of the country. This is of course nothing special about Sweden, it happens in every country. Americans just cannot stand New Yorkers (the big city) and policy makers in Washington DC. People in Russia probably hate people in Moscow. Europe is united in its disliking of the EU bureaucracy headquarters in Brussels. Talking to other Swedes of my age, it is easier to discuss individual streets in San Francisco than streets in a neighboring Swedish city where none of us have been. We know that the U.N. headquarters is on the east side of Manhattan and that Wall Street is at a walking distance from Battery Park. We all tend to look upwards. Swedes look to Stockholm, Paris, London and New York, but more seldom to Latvia or Bangladesh or Malawi.

All Swedish wikipedians know about the Swedish and the English Wikipedia. Of course the English is much larger. The fact that Ingmar Bergman's English article is three times longer (44K) than the Swedish one (16K), is taken for granted and not as an urgent crisis that needs to be addressed. The Swedish Wikipedia has articles about small Swedish places and people that don't have articles in the English Wikipedia, and so their sizes cannot be compared. It is also wellknown that the Swedish Wikipedia is slightly larger than our immediate neighboring languages Finnish, Norwegian and Danish.

The idea to compare the Swedish Wikipedia to the one in Czech, having the same number of speakers (10M), just doesn't occur, because nobody in Sweden speaks Czech and most would believe you need to know the language in order to understand anything. I guess it is mutual, the Czech speak their own language plus English and German and a bit of Polish, but nobody would care about Swedish.

It was only recently that the stubbiness of the Swedish Wikipedia, relative to other languages of Wikipedia, started to come into public awareness. Most regular wikipedians are now aware of this. But when we start to talk about improving quality, average Swedes who are occasional contributors still don't understand what this is about. They think we want to compete with traditional printed encyclopedias, and that we are losing our soul as a Net project. Why should any article be deleted, when storage is so cheap? Why require source citations and complete sentences and birth dates? Surely someone else is going to add that later.

It's easy to measure the average size of articles, but much harder to change it. Even if you add 50 bytes to each article, that would only move the average by 50 bytes and not from 2K to 4K. Just by translating from English, you can add 30K to the Swedish article on Ingmar Bergman, but that doesn't move the average size.

What I did was instead to look at the very smallest articles. There's a page [[Special:Shortpages]] for this, but unfortunately it contains both short articles and disambiguation pages. From the database dumps, I could filter out the disambiguations which are 10K on the Swedish Wikipedia, leaving 270K real articles.

I found that 0.1 percent (270 articles) were shorter than 90 bytes. This was better than the Arabic Wikipedia where 0.1 percent are shorter than 62 bytes, Latvian with 78 bytes and Estonian with 81 bytes. But it is far worse than Danish 130 bytes, Polish 160, Czech 213, German 280 and Russian 359 bytes. This was in April. By merging and improving the very shortest articles, the Swedish Wikipedia's 0.1 percent shortest articles now reach 145 bytes.

In the next step, I found 1.0 percent of articles (2700 articles) were shorter than 126 bytes, which was far worse than any other language I looked at. But during April and May this has improved to 171 bytes, which is better than Arabic and Estonian. During all of April, the Swedish Wikipedia didn't grow at all, because stubs were being merged and removed faster than new articles were written. This is when the Russian Wikipedia on May 19 got the 10th position at 283K articles.

By addressing the very shortest articles, we remove the easiest excuse for people to create new very short stubs. They can no longer point to other articles that are only 100 bytes long. After the lower limit is pushed from 120 to 140 bytes, it will continue to push upwards to 160 and 180 bytes.

Some might ask why we don't just remove everything shorter than 200 or 300 bytes. You are welcome to try to propose this, but so far every such proposal has met compact resitance. Instead I found 20 articles saying "X is an island in Y archipelago" and merged them into a nice article on "Y archipelago" with a map, so that everybody can agree that the new style is a real improvement. With that sort of aggregation, it becomes more obvious that island Z is missing or that island W doesn't really belong. Fact checking and quality assurance are improved by merging stubs. It's all about setting an example for the future. The average article size will move slowly over the next few years.

The next medium term goal is to look at the 10 percent shortest articles (27K of them), the "lower decile". For the Swedish Wikipedia, this has moved from 312 bytes in April to 349 bytes in June. For the Lithuanian, Estonian, Arabic, Danish Wikipedias it is around 400 bytes. Norwegian and Icelandic are around 500, Polish at 638. The French, Finnish and Hungarian are around 700. Latvian has 808, Czech 917, German 1081 and the Russian Wikipedia has 10 percent of its articles shorter than 1305 bytes.

If you believe the Polish 638 bytes is really bad, it's just because you haven't looked close enough at Danish and Norwegian. In fact, it's not so much worse than the French 682 bytes. You are welcome to make fun of the Swedish stubs, because it is an area where we can show constant improvement.

This hunt for "substubs" is just one of several quality improvement initiatives currently running on the Swedish Wikipedia.

Not only does the Russian Wikipedia have a high average article size (about 5500 bytes, compared with, for example, English Wikipedia at around 4100 bytes) but its graph, which has multiple peaks, seems to show that, unlike many other projects, it has more mature, medium-size articles than it does stubs.

But isn't that all Russian cyrilic letters take up 2 bytes of space? while simple latin letters only 1 byte?

My guess much of that 1400-byte will be eaten away when accounting for the cyrilic vs latin letters.

Renata

Tomasz, My impression is that you do not like the results because pl.WP has a poor ratio, that's what you initially complained about. I know - and never denied - that 50 is a small sample; I did it for 53 Wikipedias. I do have criteria, even if I did not list them up for you, you have not read my paper. But you are immediately accusing me on judging purely on feelings and attitudes to nations.

For example, at geo stubs I want at least two informations that are not bot created. http://pl.wikipedia.org/wiki/Abisynia_(powiat_bialski) I checked the first some of that Kategoria:Zalążki artykułów o polskich wsiach and only Abramy I'd count as real, because there are two informations about history (1599 and 1676). I suppose that many of the other 50,159 articles of that category are pseudo articles (they all have that part about the administrative division of 1975-1998). The same thing can be said about Kategoria:Zalążek artykułu o miejscowości francuskiej with 35,066 cities in France. Schematic Planetoid articles are no real articles, like the 14.444 in Kategoria:Planetoidy pasa głównego.

So, I can imagine why pl.WP has only 64% real articles according to my sample.

Ziko

2008/6/28 Tomasz Ganicz polimerek@gmail.com:

2008/6/28 Ziko van Dijk zvandijk@googlemail.com:

...

I have discussed my study with many people (one had similar results), but no one was so aggressive, Tomasz.

...

b)your own subconcious attitude toward various nations and Wikipedias

? Is this an accusation?

No, I am just a scientist, so I have a tendency to be sceptical and have basic knowledge about typical mistakes of doing statistical research.Too small sample, no clear criteria of evaluating it, and you did not tested the experimental error or replication of your method, by comparing results from several experiments asking other people to use your meaning of what "real" article is.

50 articles sample tested by one person, who for sure have its own attitudes is not enough to say that this or another Wikipedia is better or worse. Everyone has its own attitudes towards one or another nation. It is very natural thing. And if there is no clear definition of what is "real" article and what is not, and to evaluate this it was used google machine translation (which according to NIST survey from 2006 is found to be OK in only around 49% cases) so I am quite sure that your results cannot be taken seriously. You could have stastical error at least around 15-20% (if not more), so the results 0,60 or 0,80 is in experimental error range.

Anyway it would be interesting to make better planned experiments to evaluate the quality of Wikipedia articles, but for sure it has to be done on larger sample, some sort of "hard" criteria or a group of at least 10 researchers speaking diffrent languages and having different cultural background when to use "soft, human based" criteria.

-- Tomek "Polimerek" Ganicz http://pl.wikimedia.org/wiki/User:Polimerek http://www.ganicz.pl/poli/ http://www.ptchem.lodz.pl/en/TomaszGanicz.html

---

foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l