http://ksh.wikipedia.org/wiki/Varsseveld It's not Ripuarian (ksh), but Nedersaksisch, the text is taken directly from nds-nl. Ziko 2008/6/27 Andre Engels <andreengels(a)gmail.com>om>: -- Ziko van Dijk NL-Silvolde

Alas, judging a language edition by Wikimedia Statistics does not work. Indonesian, Asturian and Volapük WPs have the same "depth" (8), but id.WP is a very good WP. How comes? There not so many edits per article in id.WP, because it has translated a lot from English. A legitimate way to create (good) articles, but it does not need a lot of edits. Bot activity: Indeed, "bot bumps" can often easily be detected in stats tables. Especially the small Wikipedias (I suppose) show (relatively) many bot activities due to interwiki linking. On the other hand, pseudo articles can be created by hand (let a script create it outside WP and then insert it "manually"). Ziko 2008/6/27 Harel Cain <harel.cain(a)gmail.com>om>: -- Ziko van Dijk NL-Silvolde

2008/6/27 Ziko van Dijk <zvandijk(a)googlemail.com>om>: Can you explain how this evalution been done? How do you distinguish between "real" and other articles? Especially I don't believe in statiscts shown for en Wikipedia. I have a feeing that there is much more bot created articles in en Wikipedia than your statistcs show. About a year ago I wanted to evaluate the number of bot created articles created in Polish Wikipedia, and then evaluate how many of them were expanded by humans. Unfortunatelly it was impossible to perform as the bot owners do not keep records of its activity. Anyway we checked randomly what happened with bot-created articles about Polish villages and small towns, which was the very first bot produciton in our Wikikipedia. As I was strongly opposed several years ago to produce bot-created articles but failed to persuade my fellow wikipedians, I just wanted to prove that it was indeed bad idea. However, the study shown that around 70% of them were efectively expanded by humans. Villagers added quite a lot of useful stuff to these articles like histories of their villages, pictures of interesting buildings etc. Can you explain if these articles are treated "real" or "not real" in your statistics and why? -- Tomek "Polimerek" Ganicz http://pl.wikimedia.org/wiki/User:Polimerek http://www.ganicz.pl/poli/ http://www.ptchem.lodz.pl/en/TomaszGanicz.html

2008/6/28 Andre Engels <andreengels(a)gmail.com>om>: Well it is not described - I mean there is no clear criteria of evaluation mentioned. Does he speak Japanese or Polish? Is it possible to recognize "real" and "unreal" articles without understanding them? Compare: http://he.wikipedia.org/wiki/%D7%9C%D7%95%D7%93%D7%96%27 Is it "real" or "unreal" article and why? I have a feeling that it is bot created, but I am no sure about it, as I don't speak Hebrew :-) And what about this: http://uk.wikipedia.org/wiki/%D0%A4%D1%96%D0%B3%D1%83%D0%BB%D1%81_%D1%96_%D… It is quite long, but I am almost sure that it is bot created and untouch by any human, because it contains only statistical data and sentences looking as if they were machine created. I don't speak Ukrainian well but understand it a little bit. But it is still just my feelings... It is funny that this article is longer than similar in es-Wikipedia, although Spanish one was edited by humans for sure :-) http://es.wikipedia.org/wiki/F%C3%ADgols_y_Ali%C3%B1%C3%A1 and moreover - if you check all Wikipedias which contain article about Fígols i Alinyà only Spanish one looks as edited by human (but it is just my feelings I can be wrong). And this: http://ta.wikipedia.org/wiki/%E0%AE%B5%E0%AE%BE%E0%AE%B0%E0%AF%8D%E0%AE%9A%… real or not real? I really don't know, probably bot-created :-) I think if we would like to perform serios evaluation of "real" and "unreal" articles it should be based on clear, not based on "feelings" criteria, done on larger samples (at least 500 articles) and by people who understand what they are reading. -- Tomek "Polimerek" Ganicz http://pl.wikimedia.org/wiki/User:Polimerek http://www.ganicz.pl/poli/ http://www.ptchem.lodz.pl/en/TomaszGanicz.html

2008/6/28 Ziko van Dijk <zvandijk(a)googlemail.com>om>: Sorry about it, but it only shows that your results are not reliable, because it is based on your feelings and poor quality machine translations which could change in unpredictable way your feelings. I might be affraid that the results shown in your table is just a reflection of: a) the quality of machine translation performed by google - it is better for latin and germanic based languages (English, French, Italian, German, Dutch etc.) and much worse for slavic, arabic and East Asian languages. b)your own subconcious attitude toward various nations and Wikipedias - even if you are trying to evaluate them all fair Google translate produces sometimes really funny results when translating from Polish to English. For example: "Przyszłość partii przyszłością narodu" (Future of the party is the future of the nation) is translated to: "The future of the future of the nation lot" :-) Or: "Byłbym spał, gdybym mógł." ( I would sleep, if only I could) is translated to: "I would be he lay, if I only could." http://en.wikipedia.org/wiki/Machine_translation_software_usability#Trustwo… http://www.nist.gov/speech/tests/mt/2006/doc/mt06eval_official_results.html I think that a method to distunguish between "real" and "unreal" articles should be based on analysis of the history of article and formal "hard" criteria. For example one can make a criteria that if there are at least 4 sentences writen by a human it is "real article". -- Tomek "Polimerek" Ganicz http://pl.wikimedia.org/wiki/User:Polimerek http://www.ganicz.pl/poli/ http://www.ptchem.lodz.pl/en/TomaszGanicz.html

2008/6/28 Ziko van Dijk <zvandijk(a)googlemail.com>om>: No, I am just a scientist, so I have a tendency to be sceptical and have basic knowledge about typical mistakes of doing statistical research.Too small sample, no clear criteria of evaluating it, and you did not tested the experimental error or replication of your method, by comparing results from several experiments asking other people to use your meaning of what "real" article is. 50 articles sample tested by one person, who for sure have its own attitudes is not enough to say that this or another Wikipedia is better or worse. Everyone has its own attitudes towards one or another nation. It is very natural thing. And if there is no clear definition of what is "real" article and what is not, and to evaluate this it was used google machine translation (which according to NIST survey from 2006 is found to be OK in only around 49% cases) so I am quite sure that your results cannot be taken seriously. You could have stastical error at least around 15-20% (if not more), so the results 0,60 or 0,80 is in experimental error range. Anyway it would be interesting to make better planned experiments to evaluate the quality of Wikipedia articles, but for sure it has to be done on larger sample, some sort of "hard" criteria or a group of at least 10 researchers speaking diffrent languages and having different cultural background when to use "soft, human based" criteria. -- Tomek "Polimerek" Ganicz http://pl.wikimedia.org/wiki/User:Polimerek http://www.ganicz.pl/poli/ http://www.ptchem.lodz.pl/en/TomaszGanicz.html

2008/6/29 Mark Williamson <node.ue(a)gmail.com>om>: Not that recently - it's based on SYSTRAN. - d.

Tomasz Ganicz wrote: Apparently it was the 500k article event that caused Ziko to bring the topic up this time. He's frustrated (and so am I) that 500K articles is reported as an achievement, when it is indeed doubtful what quality these articles have. Still, I think he exaggerates the problem. Earlier this year, when the topic came up on meta, it was because of which languages were featured as the top 10 on www.wikipedia.org, http://meta.wikimedia.org/wiki/Top_Ten_Wikipedias Since then, the Russian Wikipedia has gained the 10th position and Swedish ("the one with all the stubs") is down to 11th, so there is one problem less to care about. During that discussion, I proposed to use the size of the compressed database dump (pages-articles.xml.bz2) as the official metric, since it both counts the total database size (one long article counts the same as two short ones) and it completely removes the impact of bot generated articles. The compressed size of the Volapük Wikipedia is very small, becase the same patterns appear in many of its numerous articles. On the talk page, there is a table where this is shown, and you can sort by column by clicking the little boxes, http://meta.wikimedia.org/wiki/Talk:Top_Ten_Wikipedias#What_problem_do_we_w… I'd like to propose a quality metric: The difference in rank between the article count and the compressed database size. The English Wikipedia is the biggest (rank 1), whether you count articles or compressed database size. So its quality is 0. The Polish Wikipedia was the 4th by article count, but the 7th by compressed database size, for a quality of 4 - 7 = -3. The Swedish Wikipedia was (when this table was compiled) the 10th biggest by article count, but the 12th biggest by compressed database size, so its quality is 10 - 12 = -2. The Russian Wikipedia was the 11th by article count, but 9th by compressed database size, so its quality is +2. This doesn't mean the Russian Wikipedia is better than the English one, only that it is better than (two of) its peers of similar size. The Volapük Wikipedia was the 15th by article count, but the worse than the 30th by compressed database size (the table is incomplete), so its quality is worse than -15. -- Lars Aronsson (lars(a)aronsson.se) Aronsson Datateknik - http://aronsson.se

2008/6/29 Stephen Bain <stephen.bain(a)gmail.com>om>: Yes, but size of avarage article can be easily icreased artifically by bot-creation of inofoboxes, navigation templates, long list of categories and interwiki etc. Take a look for example on: http://pl.wikipedia.org/wiki/Telmisartan infobox is around 90% of its content. Blame Polish Wikipedia that it allows creating such articles, I would agree with you :-) The article was created by human not by bot. Higher average size of articles in any Wikipedia can be easilly achieved just by creating bot-only articles with plenty of statstical data, huge infobox and several navigation templates. See example, I have already shown: http://uk.wikipedia.org/wiki/%D0%A4%D1%96%D0%B3%D1%83%D0%BB%D1%81_%D1%96_%D… Would be nice to have a tool comparing the "real" size of articles, by which I mean counting size of free text only - without all templates and other non-text stuff. -- Tomek "Polimerek" Ganicz http://pl.wikimedia.org/wiki/User:Polimerek http://www.ganicz.pl/poli/ http://www.ptchem.lodz.pl/en/TomaszGanicz.html

Stephen Bain wrote: In Sweden we have this problem that people in our capital are so arrogant towards the rest of the country. This is of course nothing special about Sweden, it happens in every country. Americans just cannot stand New Yorkers (the big city) and policy makers in Washington DC. People in Russia probably hate people in Moscow. Europe is united in its disliking of the EU bureaucracy headquarters in Brussels. Talking to other Swedes of my age, it is easier to discuss individual streets in San Francisco than streets in a neighboring Swedish city where none of us have been. We know that the U.N. headquarters is on the east side of Manhattan and that Wall Street is at a walking distance from Battery Park. We all tend to look upwards. Swedes look to Stockholm, Paris, London and New York, but more seldom to Latvia or Bangladesh or Malawi. All Swedish wikipedians know about the Swedish and the English Wikipedia. Of course the English is much larger. The fact that Ingmar Bergman's English article is three times longer (44K) than the Swedish one (16K), is taken for granted and not as an urgent crisis that needs to be addressed. The Swedish Wikipedia has articles about small Swedish places and people that don't have articles in the English Wikipedia, and so their sizes cannot be compared. It is also wellknown that the Swedish Wikipedia is slightly larger than our immediate neighboring languages Finnish, Norwegian and Danish. The idea to compare the Swedish Wikipedia to the one in Czech, having the same number of speakers (10M), just doesn't occur, because nobody in Sweden speaks Czech and most would believe you need to know the language in order to understand anything. I guess it is mutual, the Czech speak their own language plus English and German and a bit of Polish, but nobody would care about Swedish. It was only recently that the stubbiness of the Swedish Wikipedia, relative to other languages of Wikipedia, started to come into public awareness. Most regular wikipedians are now aware of this. But when we start to talk about improving quality, average Swedes who are occasional contributors still don't understand what this is about. They think we want to compete with traditional printed encyclopedias, and that we are losing our soul as a Net project. Why should any article be deleted, when storage is so cheap? Why require source citations and complete sentences and birth dates? Surely someone else is going to add that later. It's easy to measure the average size of articles, but much harder to change it. Even if you add 50 bytes to each article, that would only move the average by 50 bytes and not from 2K to 4K. Just by translating from English, you can add 30K to the Swedish article on Ingmar Bergman, but that doesn't move the average size. What I did was instead to look at the very smallest articles. There's a page [[Special:Shortpages]] for this, but unfortunately it contains both short articles and disambiguation pages. From the database dumps, I could filter out the disambiguations which are 10K on the Swedish Wikipedia, leaving 270K real articles. I found that 0.1 percent (270 articles) were shorter than 90 bytes. This was better than the Arabic Wikipedia where 0.1 percent are shorter than 62 bytes, Latvian with 78 bytes and Estonian with 81 bytes. But it is far worse than Danish 130 bytes, Polish 160, Czech 213, German 280 and Russian 359 bytes. This was in April. By merging and improving the very shortest articles, the Swedish Wikipedia's 0.1 percent shortest articles now reach 145 bytes. In the next step, I found 1.0 percent of articles (2700 articles) were shorter than 126 bytes, which was far worse than any other language I looked at. But during April and May this has improved to 171 bytes, which is better than Arabic and Estonian. During all of April, the Swedish Wikipedia didn't grow at all, because stubs were being merged and removed faster than new articles were written. This is when the Russian Wikipedia on May 19 got the 10th position at 283K articles. By addressing the very shortest articles, we remove the easiest excuse for people to create new very short stubs. They can no longer point to other articles that are only 100 bytes long. After the lower limit is pushed from 120 to 140 bytes, it will continue to push upwards to 160 and 180 bytes. Some might ask why we don't just remove everything shorter than 200 or 300 bytes. You are welcome to try to propose this, but so far every such proposal has met compact resitance. Instead I found 20 articles saying "X is an island in Y archipelago" and merged them into a nice article on "Y archipelago" with a map, so that everybody can agree that the new style is a real improvement. With that sort of aggregation, it becomes more obvious that island Z is missing or that island W doesn't really belong. Fact checking and quality assurance are improved by merging stubs. It's all about setting an example for the future. The average article size will move slowly over the next few years. The next medium term goal is to look at the 10 percent shortest articles (27K of them), the "lower decile". For the Swedish Wikipedia, this has moved from 312 bytes in April to 349 bytes in June. For the Lithuanian, Estonian, Arabic, Danish Wikipedias it is around 400 bytes. Norwegian and Icelandic are around 500, Polish at 638. The French, Finnish and Hungarian are around 700. Latvian has 808, Czech 917, German 1081 and the Russian Wikipedia has 10 percent of its articles shorter than 1305 bytes. If you believe the Polish 638 bytes is really bad, it's just because you haven't looked close enough at Danish and Norwegian. In fact, it's not so much worse than the French 682 bytes. You are welcome to make fun of the Swedish stubs, because it is an area where we can show constant improvement. This hunt for "substubs" is just one of several quality improvement initiatives currently running on the Swedish Wikipedia. -- Lars Aronsson (lars(a)aronsson.se) Aronsson Datateknik - http://aronsson.se

But isn't that all Russian cyrilic letters take up 2 bytes of space? while simple latin letters only 1 byte? My guess much of that 1400-byte will be eaten away when accounting for the cyrilic vs latin letters. Renata

Tomasz, My impression is that you do not like the results because pl.WP has a poor ratio, that's what you initially complained about. I know - and never denied - that 50 is a small sample; I did it for 53 Wikipedias. I do have criteria, even if I did not list them up for you, you have not read my paper. But you are immediately accusing me on judging purely on feelings and attitudes to nations. For example, at geo stubs I want at least two informations that are not bot created. http://pl.wikipedia.org/wiki/Abisynia_(powiat_bialski) I checked the first some of that Kategoria:Zalążki artykułów o polskich wsiach and only Abramy I'd count as real, because there are two informations about history (1599 and 1676). I suppose that many of the other 50,159 articles of that category are pseudo articles (they all have that part about the administrative division of 1975-1998). The same thing can be said about Kategoria:Zalążek artykułu o miejscowości francuskiej with 35,066 cities in France. Schematic Planetoid articles are no real articles, like the 14.444 in Kategoria:Planetoidy pasa głównego. So, I can imagine why pl.WP has only 64% real articles according to my sample. Ziko 2008/6/28 Tomasz Ganicz <polimerek(a)gmail.com>om>: -- Ziko van Dijk NL-Silvolde