Re: [discovery] Multilingual and cross-project search

Hi Pine, Thanks for reaching out with your questions. I'm hoping that what I've written below will give you some answers of what we're working on. *tl;dr - we're working on it ;)* The Discovery Search team have been quite busy over the last few quarters working on getting the backend quicker and more accurate in serving up search results with our updated completion suggester <https://www.mediawiki.org/wiki/Extension:CirrusSearch/CompletionSuggester> and it really does show on our dashboard for zero results <http://discovery.wmflabs.org/metrics/#failure_rate> returned. However, we still have lots of work to do, including updating the elasticsearch cluster <https://phabricator.wikimedia.org/T136001> and logstash <https://phabricator.wikimedia.org/T136003>. We've got a quite a few tickets in Phabricator to investigate how we can do searches across projects <https://phabricator.wikimedia.org/T137312> and languages. One such ticket <https://phabricator.wikimedia.org/T138958> is to detect if the user used the 'wrong' keyboard when typing in their search on the en/ru Wikipedias using a Russian/American keyboard. We also have a TextCat demo <https://tools.wmflabs.org/textcatdemo/> that you might want to check out and using wikidata aliases and descriptions <https://phabricator.wikimedia.org/T117483> are also generally being looked at. Additionally, on our backlog <https://phabricator.wikimedia.org/tag/discovery-search-backlog/> is to improve 'fuzzy' full text <https://phabricator.wikimedia.org/T135920> searches to return more meaningful results. Because the autocomplete and full text algorithms are very different, this can cause odd results in some cases. One option we'll can take a look at is if the full text algorithm doesn't return any results, run that same search through the completion suggester. It would take additional time on the backend of course, to run this additional search with the added difficulty in that the completion suggester doesn't run the elasticsearch highlighter. We also are looking at how to handle question marks (?) in a search query and to design a page that would display search results, across languages, in a meaningful manner to our user, that might be able to show redirects better as well. Whew...lots going on, so stay tuned! Cheers, Deb -- Deb Tankersley Product Manager, Discovery IRC: debt Wikimedia Foundation On Mon, Jul 4, 2016 at 9:35 AM, Pine W &lt;wiki.pine(a)gmail.com&gt; wrote: _______________________________________________ discovery mailing list discovery(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery