Hi Team,
I often see missed results in the Quick Search input box when phrases sometimes are indexed with an apostrophe, but its missed because we sometimes just type simple titles or simple names like so:
"no mans sky" (no results)
but when I click "containing... no mans sky" then the results page displays https://www.wikidata.org/w/index.php?search=no%20mans%20sky&title=Special%3ASearch&fulltext=1&ns0=1&ns8=1&ns14=1&ns120=1&ns122=1&ns146=1&ns640=1&ns828=1&ns1198=1 and shows the video game correctly.
It seems this might be because of an indexing choice for the Quick Search input box? (I know that Lucene-Solr can handle this for English input preference https://github.com/apache/lucene-solr/blob/master/lucene/analysis/common/src... )
Hi,
On Tue, Nov 5, 2019 at 9:15 PM Thad Guidry thadguidry@gmail.com wrote:
Hi Team,
I often see missed results in the Quick Search input box when phrases sometimes are indexed with an apostrophe, but its missed because we sometimes just type simple titles or simple names like so:
"no mans sky" (no results)
but when I click "containing... no mans sky" then the results page displays https://www.wikidata.org/w/index.php?search=no%20mans%20sky&title=Special%3ASearch&fulltext=1&ns0=1&ns8=1&ns14=1&ns120=1&ns122=1&ns146=1&ns640=1&ns828=1&ns1198=1 and shows the video game correctly.
It seems this might be because of an indexing choice for the Quick Search input box? (I know that Lucene-Solr can handle this for English input preference https://github.com/apache/lucene-solr/blob/master/lucene/analysis/common/src... )
The reason you find "no man's sky" when searching "no mans sky" using Special:Search is simply because "mans" is matched somewhere in the document (could be because of the stemmer or for example the Quora topic id No-*Mans*-Sky-5).
The ApostropheFilter that you mention unfortunately has been designed for the turkish language and would remove everything after the apostrophe which is not what we want in general.
Completion (quick search input box) and Special:Search use very different techniques, what we do for apostrophe in completion is transforming it into a space. Should we do something different for apostrophes in completion search? I have no clue and this would require some evaluation, if you believe this is important as it causes much annoyance feel free to file a task in phabricator so that we can discuss about it.
David.
DONE !
https://phabricator.wikimedia.org/T237645
Thad https://www.linkedin.com/in/thadguidry/
On Thu, Nov 7, 2019 at 3:14 AM David Causse dcausse@wikimedia.org wrote:
Hi,
On Tue, Nov 5, 2019 at 9:15 PM Thad Guidry thadguidry@gmail.com wrote:
Hi Team,
I often see missed results in the Quick Search input box when phrases sometimes are indexed with an apostrophe, but its missed because we sometimes just type simple titles or simple names like so:
"no mans sky" (no results)
but when I click "containing... no mans sky" then the results page displays https://www.wikidata.org/w/index.php?search=no%20mans%20sky&title=Special%3ASearch&fulltext=1&ns0=1&ns8=1&ns14=1&ns120=1&ns122=1&ns146=1&ns640=1&ns828=1&ns1198=1 and shows the video game correctly.
It seems this might be because of an indexing choice for the Quick Search input box? (I know that Lucene-Solr can handle this for English input preference https://github.com/apache/lucene-solr/blob/master/lucene/analysis/common/src... )
The reason you find "no man's sky" when searching "no mans sky" using Special:Search is simply because "mans" is matched somewhere in the document (could be because of the stemmer or for example the Quora topic id No-*Mans*-Sky-5).
The ApostropheFilter that you mention unfortunately has been designed for the turkish language and would remove everything after the apostrophe which is not what we want in general.
Completion (quick search input box) and Special:Search use very different techniques, what we do for apostrophe in completion is transforming it into a space. Should we do something different for apostrophes in completion search? I have no clue and this would require some evaluation, if you believe this is important as it causes much annoyance feel free to file a task in phabricator so that we can discuss about it.
David. _______________________________________________ Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
wikidata-tech@lists.wikimedia.org