[Wikitech-l] Hyper Estraier extension

Hello, I wrote a MediaWiki extension to search wikicontent using Hyper Estraier, a full-text search engine. Description has been posted at http://meta.wikimedia.org/wiki/Hyper_Estraier_extension About Hyper Estraier: * Sourceforge: http://hyperestraier.sourceforge.net/ * OS Reviews: http://www.osreviews.net/reviews/misc/hyperestraier And I set up a demo site uses this extension to search Japanese Wikipedia content. http://ja.wikipedia.tietew.jp/wiki/Special:Search Look'n'feel is derived from LuceneSearch.php. Search results and content summary are generated by Hyper Estraier search server. == Background == Current wikipedia's lucene-search is tooo bad about searching Japanese. I think Hyper Estraier is better for languages whose sentence is not separated by space, such as Japanese and Chinese. see also: http://meta.wikimedia.org/wiki/Summer_of_Code_2006#I18n_search_index ## I have no idea which engine is better for europian languages. == Request == I want to test and demo real-time index update from live Wikipedia. Could you please give me a permittion to access OAIRepository? == Comparation == word: "ウィキペディア" (Wikipedia) Hyper Estraier: 490 hits http://ja.wikipedia.tietew.jp/w/index.php?title=%E7%89%B9%E5%88%A5:Search&a… http://ja.wikipedia.tietew.jp/w/index.php?title=%E7%89%B9%E5%88%A5:Search&a… Lucene: 397 hits ... but only 40 documents are shown http://ja.wikipedia.org/w/index.php?title=%E7%89%B9%E5%88%A5:Search&sea… http://ja.wikipedia.org/w/index.php?title=%E7%89%B9%E5%88%A5:Search&sea… word: "百科事典" (hyakka-jiten; encyclopedia) Hyper Estraier: 352 hits http://ja.wikipedia.tietew.jp/w/index.php?title=%E7%89%B9%E5%88%A5:Search&a… http://ja.wikipedia.tietew.jp/w/index.php?title=%E7%89%B9%E5%88%A5:Search&a… Lucene: 171 hits ... but only 23 documents are shown http://ja.wikipedia.org/wiki/%E7%89%B9%E5%88%A5:Search?search=%E7%99%BE%E7%… http://ja.wikipedia.org/wiki/%E7%89%B9%E5%88%A5:Search?search=%E7%99%BE%E7%… Thank you. -- [[User:Tietew]] <tietew(a)tietew.net> Wiki: http://meta.wikimedia.org/wiki/User:Tietew PGP/GPG: 26CB 71BB B595 09C4 0153 81C4 773C 963A D51B 8CAA