Forwarding to the Discovery mailing list, although it sounds like the OP hopes to have any possible discussion on Wikitech-l. I wonder if there would be ways for WMF Discovery to leverage the work that's being done already on Commoncrawl and Commonsearch for use in Wikimedia internal search. Pine ---------- Forwarded message ---------- From: Sylvain Zimmer <sylvain(a)sylvainzimmer.com> Date: Sun, Mar 6, 2016 at 11:46 AM Subject: [Wikitech-l] Using Wikipedia/Wikidata in a nonprofit search engine To: wikitech-l(a)lists.wikimedia.org Hi, Some of you may be familiar with http://commoncrawl.org ; they are doing an excellent job of making large crawls of the web accessible to everyone. I've been working on an open search engine based on these crawls for a while, and I would love to have your feedbacks on the project: https://about.commonsearch.org/ Specifically, I would be curious to know what you would consider to be the best possible integration of Wikipedia & Wikidata in a general search engine? As a first step, we have just started using the "official website" property from Wikidata and we are considering importing the Wikipedia abstracts next (https://github.com/commonsearch/cosr-back/issues/11). I'm looking forward to your feedbacks... or contributions! :-) Thanks in advance, PS: A few wikimedians recommended me to post on wikitech-l to keep the focus on the technical aspects of the project and hopefully avoid linking this project in any way to the KE stuff, which it actually predates by far (https://news.ycombinator.com/item?id=6209088). -- Sylvain Zimmer http://sylvinus.org _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l