zhengzhu wrote: Fantastic work! Regarding the word segmentation problem: Googling for "Chinese word segmentation" gives a large number of references to various interesting research on this field. There seems to be a lively research community in this field: perhaps some of them might want to earn kudos points by collaborating on improving the performance of this module by tackling some of the common segmentation issues. Dynamic programming and Hidden Markov methods seem to be popular, and we must, by now, have a quite large corpus of our own in the zh: wikipedia. -- Neil

On 4/13/05, Milos Rancic <millosh(a)gmail.com> wrote: My English is bad... Sentecne "No one should be forced to use markup, but there are some people at sr: who woudl do that." should be "No one should be forced to use markup, but there are some people at sr: who would markup texts." :)

On 4/13/05, Ray Saintonge <saintonge(a)telus.net> wrote: Hehehe. I understand that situation with Serbian standard variants is funny to others, but let me explain :) Father of modern Serbian standard language is Vuk Stefanovic Karadzic. He was iyekavian by origin. He was borned at iyekavian part of western Serbia. (Today, that part of Serbia is ekavian because of Belgrade-oriented centralization of Serbia.) But, the center of Serbian culture was not in Serbia (which became independent at the first half of 19th century). It was in Vojvodina, which was the part of Austro-Hungary empire. And, people from Vojvodina, but from Belgrade, too, was (and are) speaking ekavian. One more but: Serbs from Montenegro, eastern Herzegovina and Bosnia was (and are) iyekavian. So, before Djuro Danicic (Vuk's student) introduced new Latin alphabet (at the first time completely adopted by Croats, today only letter đ/Đ is from original Danicic's alphabet), Vuk and Corat Ljudevit Gaj introduced "two variants of the same language". And, that language was called with different names: "Croatian or Serbian", "Serbian or Croatian", "Serbo-Croatian" and "Croato-Serbian". As centralistic communits system didn't want to devide language to "Croatian" and "Serbian", they devided language just into iyekavian and ekavian. So, Serbs from Bosnia, Herzegovina and Montenegro was speaking the same variant of language with Croats, while Serbs from Serbia was speaking another variant. Today, standards are Serbian, Croatian and Bosnian. When Former Yugoslavia was destroyed, as well as standard Serbo-Croatian language, Croats and Bosniaks had clear situation: The big majority of Croats and Bosniaks are iyekavian. However, Serbs didn't have that situation: some Serbs are ekavian and some Serbs are iyekavian. During the war in Bosnia, there were one linguistic experiment (supported by low, military and police), which was introduced by Radovan Karadzic regime: Standard language became ekavian. All of radio, television, newspapers and state authorities was forced to write in ekavian. (In very polite words, I think that it was not good idea.) Today, organization which takes care about Serbian standard is "Council for Standardization of Serbian Language", which has delegates from academic and government institution from Serbia, Republic of Srpska and Montenegro. And, the only part of Serbian language politics which has almost concensus is that Serbian language has two standards: ekavian and iyekavian. Alphabets are the problem in politics, standards are not. Children, tinagers and students from Republic of Srpska and Montenegro are learning iyekavian variant and differences in texts are bigger then in two main variants of English language. You can have a lot of articles in English without diferences like "kilometer-kilometre" or such. But, it is probabbly that you will find if the text is iyekavian or ekavian in the first sentence of article. However, there are no misunderstandings between iyekavian and ekavian speaker. As I see, implementation of ekavian/iyekavian is more easy then implementation of transliteration between Cyrillic and Latin alphabet and vice versa. And I am sure that you can keep variants as one text inside of database. Also, I don't care about Serbian nationalism. I care about live attributes of Serbian culture. And Wikipedia is the place which can implement them. If people here have good will.

Kaixo! (I lost the thread, so I'm replying here) On Wed, Apr 13, 2005 at 10:55:14PM +0200, Milos Rancic wrote: is that difference only with the letter "e"? If yes, why can't it be decided that some people pronounce it as "je" (like they do in Russian, they just write "е" and not "йе") and some other people pronounce it as "e", and keep a single way of writting? or the other way around, always write the "j", but some people not pronounce it. having two different writting systems just for that (which means four different writting systemns in total) seems unjustified; -- Ki ça vos våye bén, Pablo Saratxaga http://chanae.walon.org/pablo/ PGP Key available, key ID: 0xD9B85466 [you can write me in Walloon, Spanish, French, English, Catalan or Esperanto] [min povas skribi en valona, esperanta, angla aux latinidaj lingvoj]

There are some more differences... In 12th century so called "new dialects of Serbian/Croatian language" are devided over different reflexes of old vocal "jat" (capital letter "Ѣ", small letter "ѣ" in Cyrillic, or Ě / ě in Latin) into: ekavian, iyekavian and ikavian variant. Ekavian reflex is almost always "e" (sometimes it is "i"), ikavian reflex is always "i", and iyekavian reflexes are in general "ye" or "iye", but often "e" and rare "i". After 12th century there some consonant changes was happened. When iyekavian reflex "ye" stayed after consonants 'l' or 'n', string "lje" (lye) or "nje" (nye) became two, not three volwes and "lj" and "nj" was not anymore "l+j" or "n+j", but "lj" and "nj" as soft "l" and "n". So, differences became bigger, but not so big. Ekavians and iyekavians are closer between themselfs then with any other language speakers. For example, when iyekavian Croat from Zagreb or iyekavian Serb from Banja Luka talks with ekavian Serb from Belgrade, they will not have any problem in communication. However, in writting systems there are some differences. Not only "mleko-mlijeko", but, for example, word for "notepad" in ekavian variant is "beležnica", but in iyekavian it is "bilježnica". Old form was "bělěžnica" (before "jat" transformation, after other changes in language). Ekavians changed "ě" into e, but iyekavians changed first "ě" into 'i' and second into "ye". As /j/ made 'l' soft, "lj" combination dosn't represent two volwes, but one. Around 20.000 lexemes (and around 200.000 forms/words) are affected with "jat" transformation. Vuk Karadzic and Ljudevit Gaj sancionied only ekavian and iyekavian, but not ikavian. So, today Croatian and Bosnian have only iyekavian varian (a lot of Croats and some of Bosniaks are ikavian) and Serbian has iyekavian and ekavian. There are a lot of possibilities for solving that problem. However, you and me are not the persons who are making those decisions. Also, I think that a lot of Serbian linguists don't think a lot about complicated situation with standards. A lot of them would use Cyrillic or Latin (or both if they don't care about alphabet) and one dialect. Ortography of Serbian Language is written only in Cyrillic in two variants (ekavian and iyekavian)... The other question is that Serbian (Croatian and Bosnian) language have (with Finish and, I think, Estonian) the most phonetic alphabets. This is two centuries tradition of Serbian language and I am the first who would be against morphological writing system (as well as other linguists). Morphological solution is introduction of "jat" into ortography (Ěě in Latin, Ѣѣ in Cyrillic), but a lot of Ekavians would not know where to put it (not all 'e' came from 'jat'); actualy, only Ekavians with high linguistic education know where to put 'jat' and where not to put. About other way, I said that Karadzic's nazi regime tried to do that. I can say that cultures are complicated. They are not based on bits and formal logic. Hindi and Urdu are the same linguistic language, as well and Serbian, Croatian and Bosnian. However, those are different cultures; and all of that cultures has set of their own rules. 10 million Serbs has four writting systems. But, 500.000 of Lusatian Serbs has two writting systems :) Or, almost 5 million inhabitans of Papua New Guinea has around 800 different (!) languages!

Thanks Heli, it looks nice and very useful ! Tom Heli Retzek schrieb: