Hi,
I hope this is the right list to post this email - otherwise I would appreciate being directed to the right one.
Shortly, I would like to promote a project for opening the search box to external entities. My main motivation would be shared by many researchers in interactive information retrieval (IIR): In order to run experiments about new techniques in IIR, it is necessary to evaluate them, and hence to have enough users to try new approaches. It is possible to simulate or to do low scale experiments, but to validate such approaches necessitate much bigger databases.
My proposal would be to include a third option below the search box, which would be to use an external search engine which would communicate with wikipedia in order to provide search results - the communication would allow wikipedia to control what is happening in order to avoid problems (from latency to spam).
The search box would allow a user to use either a "random" search engine, or to use one that could be set in the preferences.
I would suggest the randomness to be not so random, in the sense that it should favour good search engines over bad one - hence the title "Darwinian search". That would improve the special search box quality over time, while stimulating research in my area.
I think it would also be beneficial for wikipedia, since 1) it distributes the search load to other back ends 2) it would improve search quality (and may change the way people use wikipedia) and may be included as a default by wikipedia in the longer term 3) it does not cost much - once the API and the main means to ensure quality are set, the system will work by itself
I do not develop more here, since I first want to know if there is some interest.
Best regards, Benjamin Piwowarski (University of Glasgow, UK)
2009/5/26 Benjamin Piwowarski benjamin@bpiwowar.net:
I do not develop more here, since I first want to know if there is some interest.
At the present time there is a slight lack of open source search engines and people interested in working on them would probably be better of submitting patches to mediawiki's search code.
On 26 May 2009, at 13:20, geni wrote:
2009/5/26 Benjamin Piwowarski benjamin@bpiwowar.net:
I do not develop more here, since I first want to know if there is some interest.
At the present time there is a slight lack of open source search engines and people interested in working on them would probably be better of submitting patches to mediawiki's search code.
Hi,
I guess it will be harder to make people (at least from university) interested in submitting patches to an existing software than to provide a way for them to plug-in their search engines. I would at least be more interested to work that way, because it is simpler and some approaches need more than patches to be implemented. I understand that it may not fit the interests of wikipedia in general, but I think it could if done properly by stimulating people to submit alternative approaches.
Benjamin
To make sure I'm understanding you correctly... You would like to add academic or other (non-major) search engines to the drop down box on Special:Search on the English Wikipedia? That currently allows searching via Google, Yahoo, Windows Live, Wikiwix and Exalead in addition to the MediaWiki search.
Probably the best list for this sort of discussion is wikitech-l.
Nathan
On 26 May 2009, at 14:43, Nathan wrote:
To make sure I'm understanding you correctly... You would like to add academic or other (non-major) search engines to the drop down box on Special:Search on the English Wikipedia? That currently allows searching via Google, Yahoo, Windows Live, Wikiwix and Exalead in addition to the MediaWiki search.
Yes, that would be a good starting idea, although it would be nice to see it as an option for all the searches (i.e. using the search box that appears on all the pages of wikipedia) - but may be this can be done latter.
Probably the best list for this sort of discussion is wikitech-l.
OK, I will write to this list, thanks.
Thanks Benjamin
Some may find this interesting.
http://www.cbc.ca/technology/story/2009/05/22/tech-vancouver-open-source-sta...
Ec
wikimedia-l@lists.wikimedia.org