So I was looking up information on peripheral neuritis[0] and I accidentally mistyped it as "peripheral neuriti". The good news: the autocorrector worked out I'd done it wrong, corrected it, and sent me automatically to the right results. Yay![1]
But looking at the results I see a really obvious improvement we could make that would definitely improve the user experience in this scenario. See, if you look at the first article on the list you'll see it's "Peripheral neuropathy". Why? Because peripheral neuritis redirects to that. But the article header appears in the search results as "Peripheral neuropathy", since that's the real title.
But it's not what I searched for. What I searched for was neuritis. Is neuritis the same as neuropathy? I dunno, I'm a random reader. Is this a good search result to click on? No idea.
What I'd love for us to do is run an A/B test with two conditions:
1. Users who search for a term which redirects to an article get the current experience (control) 2. Users who search for a term which redirects to an article get the article title in the search results claiming to be the redirect title (test)
I bet this would really improve the clickthrough rate for this class of searches. It would definitely improve the UX.
[0] I'm researching thalidomide. Long story. [1] https://en.wikipedia.org/w/index.php?search=peripheral+neuriti&title=Spe...
Though it isn't exactly the same, this reminds me of our discussion back in Aug/Sep about Corn and Maize, much of which is on Phabricator: https://phabricator.wikimedia.org/T110571 .
The behavior you want happens sometimes. Searching for "usually" gives as the first result: Convention (norm) (redirect from Usually)
"difficult": Difficulty (redirect from Difficult)
"corn", among its many odd behaviors, gives: Maize (redirect from Maize corn).
I think there's some sort of ranking going on for redirects that can put a partial match above an exact match (hence "Maize corn" instead of "corn" as the redirect shown).
More relevant to your example, I think that if there's a match on one or more search terms in the redirected-to title, it blocks redirected-from titles from being shown. Searching for "neropathic" gets a spelling correction to "neuropathic", which gives as the first result: Peripheral neuropathy (redirect from Neuropathic).
David may know more about either of these cases off the top of his head. (But maybe not, since it was a while ago, and I recall that it was very messy, and we declined to work on it then.)
So, at the very least, the control group would at least sometimes have a chance of getting the test behavior because we do that now under certain circumstances.
Another option is to consider this a bug fix so that we have consistent behavior whenever there's an exact match of the query to a redirect title, regardless of partial matches to the redirected-to title or higher-ranked redirects.
It would also be interesting to know how often queries get redirected results, and how often they show the redirects—either in general, or as part of any A/B test.
—Trey
Trey Jones Software Engineer, Discovery Wikimedia Foundation
On Sat, Nov 28, 2015 at 3:40 AM, Oliver Keyes okeyes@wikimedia.org wrote:
So I was looking up information on peripheral neuritis[0] and I accidentally mistyped it as "peripheral neuriti". The good news: the autocorrector worked out I'd done it wrong, corrected it, and sent me automatically to the right results. Yay![1]
But looking at the results I see a really obvious improvement we could make that would definitely improve the user experience in this scenario. See, if you look at the first article on the list you'll see it's "Peripheral neuropathy". Why? Because peripheral neuritis redirects to that. But the article header appears in the search results as "Peripheral neuropathy", since that's the real title.
But it's not what I searched for. What I searched for was neuritis. Is neuritis the same as neuropathy? I dunno, I'm a random reader. Is this a good search result to click on? No idea.
What I'd love for us to do is run an A/B test with two conditions:
- Users who search for a term which redirects to an article get the
current experience (control) 2. Users who search for a term which redirects to an article get the article title in the search results claiming to be the redirect title (test)
I bet this would really improve the clickthrough rate for this class of searches. It would definitely improve the UX.
[0] I'm researching thalidomide. Long story. [1] https://en.wikipedia.org/w/index.php?search=peripheral+neuriti&title=Spe... -- Oliver Keyes Count Logula Wikimedia Foundation
discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery
Agreed on the logging, and on the options. I don't think I'd call it a bug fix quuuite yet because it's a not-insignificant change, in terms of effect, for users. I'd much rather test it than have to wildly hypothesise if our numbers change.
On 30 November 2015 at 09:50, Trey Jones tjones@wikimedia.org wrote:
Though it isn't exactly the same, this reminds me of our discussion back in Aug/Sep about Corn and Maize, much of which is on Phabricator: https://phabricator.wikimedia.org/T110571 .
The behavior you want happens sometimes. Searching for "usually" gives as the first result: Convention (norm) (redirect from Usually)
"difficult": Difficulty (redirect from Difficult)
"corn", among its many odd behaviors, gives: Maize (redirect from Maize corn).
I think there's some sort of ranking going on for redirects that can put a partial match above an exact match (hence "Maize corn" instead of "corn" as the redirect shown).
More relevant to your example, I think that if there's a match on one or more search terms in the redirected-to title, it blocks redirected-from titles from being shown. Searching for "neropathic" gets a spelling correction to "neuropathic", which gives as the first result: Peripheral neuropathy (redirect from Neuropathic).
David may know more about either of these cases off the top of his head. (But maybe not, since it was a while ago, and I recall that it was very messy, and we declined to work on it then.)
So, at the very least, the control group would at least sometimes have a chance of getting the test behavior because we do that now under certain circumstances.
Another option is to consider this a bug fix so that we have consistent behavior whenever there's an exact match of the query to a redirect title, regardless of partial matches to the redirected-to title or higher-ranked redirects.
It would also be interesting to know how often queries get redirected results, and how often they show the redirects—either in general, or as part of any A/B test.
—Trey
Trey Jones Software Engineer, Discovery Wikimedia Foundation
On Sat, Nov 28, 2015 at 3:40 AM, Oliver Keyes okeyes@wikimedia.org wrote:
So I was looking up information on peripheral neuritis[0] and I accidentally mistyped it as "peripheral neuriti". The good news: the autocorrector worked out I'd done it wrong, corrected it, and sent me automatically to the right results. Yay![1]
But looking at the results I see a really obvious improvement we could make that would definitely improve the user experience in this scenario. See, if you look at the first article on the list you'll see it's "Peripheral neuropathy". Why? Because peripheral neuritis redirects to that. But the article header appears in the search results as "Peripheral neuropathy", since that's the real title.
But it's not what I searched for. What I searched for was neuritis. Is neuritis the same as neuropathy? I dunno, I'm a random reader. Is this a good search result to click on? No idea.
What I'd love for us to do is run an A/B test with two conditions:
- Users who search for a term which redirects to an article get the
current experience (control) 2. Users who search for a term which redirects to an article get the article title in the search results claiming to be the redirect title (test)
I bet this would really improve the clickthrough rate for this class of searches. It would definitely improve the UX.
[0] I'm researching thalidomide. Long story. [1] https://en.wikipedia.org/w/index.php?search=peripheral+neuriti&title=Spe... -- Oliver Keyes Count Logula Wikimedia Foundation
discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery
discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery
It looks like a bug...
As Trey said it happens sometimes. But for peripheral neuritis it's not the case, the highlighter decided to highlight a partial match ( *peripheral* neuropathy ) instead of the full match to the redirect. I think that when there's a single match to a word in the title any other matches (even better ones) to redirects, categories or headings will be ignored.
We will have to review the highlighter configuration to fix this issue.
Le 30/11/2015 15:50, Trey Jones a écrit :
Though it isn't exactly the same, this reminds me of our discussion back in Aug/Sep about Corn and Maize, much of which is on Phabricator: https://phabricator.wikimedia.org/T110571 .
The behavior you want happens sometimes. Searching for "usually" gives as the first result: Convention (norm) (redirect from Usually)
"difficult": Difficulty (redirect from Difficult)
"corn", among its many odd behaviors, gives: Maize (redirect from Maize corn).
I think there's some sort of ranking going on for redirects that can put a partial match above an exact match (hence "Maize corn" instead of "corn" as the redirect shown).
More relevant to your example, I think that if there's a match on one or more search terms in the redirected-to title, it blocks redirected-from titles from being shown. Searching for "neropathic" gets a spelling correction to "neuropathic", which gives as the first result: Peripheral neuropathy (redirect from Neuropathic).
David may know more about either of these cases off the top of his head. (But maybe not, since it was a while ago, and I recall that it was very messy, and we declined to work on it then.)
So, at the very least, the control group would at least sometimes have a chance of getting the test behavior because we do that now under certain circumstances.
Another option is to consider this a bug fix so that we have consistent behavior whenever there's an exact match of the query to a redirect title, regardless of partial matches to the redirected-to title or higher-ranked redirects.
It would also be interesting to know how often queries get redirected results, and how often they show the redirects—either in general, or as part of any A/B test.
—Trey
Trey Jones Software Engineer, Discovery Wikimedia Foundation
On Sat, Nov 28, 2015 at 3:40 AM, Oliver Keyes <okeyes@wikimedia.org mailto:okeyes@wikimedia.org> wrote:
So I was looking up information on peripheral neuritis[0] and I accidentally mistyped it as "peripheral neuriti". The good news: the autocorrector worked out I'd done it wrong, corrected it, and sent me automatically to the right results. Yay![1] But looking at the results I see a really obvious improvement we could make that would definitely improve the user experience in this scenario. See, if you look at the first article on the list you'll see it's "Peripheral neuropathy". Why? Because peripheral neuritis redirects to that. But the article header appears in the search results as "Peripheral neuropathy", since that's the real title. But it's not what I searched for. What I searched for was neuritis. Is neuritis the same as neuropathy? I dunno, I'm a random reader. Is this a good search result to click on? No idea. What I'd love for us to do is run an A/B test with two conditions: 1. Users who search for a term which redirects to an article get the current experience (control) 2. Users who search for a term which redirects to an article get the article title in the search results claiming to be the redirect title (test) I bet this would really improve the clickthrough rate for this class of searches. It would definitely improve the UX. [0] I'm researching thalidomide. Long story. [1] https://en.wikipedia.org/w/index.php?search=peripheral+neuriti&title=Special%3ASearch&go=Go -- Oliver Keyes Count Logula Wikimedia Foundation _______________________________________________ discovery mailing list discovery@lists.wikimedia.org <mailto:discovery@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/discovery
discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery
Yeah, absolutely. if the user's search string, after autocorrection, _perfectly matches a redirect_ they should get the redirect as the first search result.
On 30 November 2015 at 10:30, David Causse dcausse@wikimedia.org wrote:
It looks like a bug...
As Trey said it happens sometimes. But for peripheral neuritis it's not the case, the highlighter decided to highlight a partial match ( *peripheral* neuropathy ) instead of the full match to the redirect. I think that when there's a single match to a word in the title any other matches (even better ones) to redirects, categories or headings will be ignored.
We will have to review the highlighter configuration to fix this issue.
Le 30/11/2015 15:50, Trey Jones a écrit :
Though it isn't exactly the same, this reminds me of our discussion back in Aug/Sep about Corn and Maize, much of which is on Phabricator: https://phabricator.wikimedia.org/T110571 .
The behavior you want happens sometimes. Searching for "usually" gives as the first result: Convention (norm) (redirect from Usually)
"difficult": Difficulty (redirect from Difficult)
"corn", among its many odd behaviors, gives: Maize (redirect from Maize corn).
I think there's some sort of ranking going on for redirects that can put a partial match above an exact match (hence "Maize corn" instead of "corn" as the redirect shown).
More relevant to your example, I think that if there's a match on one or more search terms in the redirected-to title, it blocks redirected-from titles from being shown. Searching for "neropathic" gets a spelling correction to "neuropathic", which gives as the first result: Peripheral neuropathy (redirect from Neuropathic).
David may know more about either of these cases off the top of his head. (But maybe not, since it was a while ago, and I recall that it was very messy, and we declined to work on it then.)
So, at the very least, the control group would at least sometimes have a chance of getting the test behavior because we do that now under certain circumstances.
Another option is to consider this a bug fix so that we have consistent behavior whenever there's an exact match of the query to a redirect title, regardless of partial matches to the redirected-to title or higher-ranked redirects.
It would also be interesting to know how often queries get redirected results, and how often they show the redirects—either in general, or as part of any A/B test.
—Trey
Trey Jones Software Engineer, Discovery Wikimedia Foundation
On Sat, Nov 28, 2015 at 3:40 AM, Oliver Keyes okeyes@wikimedia.org wrote:
So I was looking up information on peripheral neuritis[0] and I accidentally mistyped it as "peripheral neuriti". The good news: the autocorrector worked out I'd done it wrong, corrected it, and sent me automatically to the right results. Yay![1]
But looking at the results I see a really obvious improvement we could make that would definitely improve the user experience in this scenario. See, if you look at the first article on the list you'll see it's "Peripheral neuropathy". Why? Because peripheral neuritis redirects to that. But the article header appears in the search results as "Peripheral neuropathy", since that's the real title.
But it's not what I searched for. What I searched for was neuritis. Is neuritis the same as neuropathy? I dunno, I'm a random reader. Is this a good search result to click on? No idea.
What I'd love for us to do is run an A/B test with two conditions:
- Users who search for a term which redirects to an article get the
current experience (control) 2. Users who search for a term which redirects to an article get the article title in the search results claiming to be the redirect title (test)
I bet this would really improve the clickthrough rate for this class of searches. It would definitely improve the UX.
[0] I'm researching thalidomide. Long story. [1] https://en.wikipedia.org/w/index.php?search=peripheral+neuriti&title=Spe... -- Oliver Keyes Count Logula Wikimedia Foundation
discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery
discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery
discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery
Hi!
But it's not what I searched for. What I searched for was neuritis. Is neuritis the same as neuropathy? I dunno, I'm a random reader. Is this a good search result to click on? No idea.
What I'd love for us to do is run an A/B test with two conditions:
- Users who search for a term which redirects to an article get the
current experience (control) 2. Users who search for a term which redirects to an article get the article title in the search results claiming to be the redirect title (test)
I bet this would really improve the clickthrough rate for this class of searches. It would definitely improve the UX.
Sometimes I when I look for redirect, like "Obamma", I get this:
Barack Obama (redirect from Obamma)
Which seems to be the best way of presenting it, it both says what I looked for (even if it's wrong) and what I actually should have looked for (which is educational). Maybe it should show that always if it's redirect-driven?
On 30 November 2015 at 13:01, Stas Malyshev smalyshev@wikimedia.org wrote:
Hi!
But it's not what I searched for. What I searched for was neuritis. Is neuritis the same as neuropathy? I dunno, I'm a random reader. Is this a good search result to click on? No idea.
What I'd love for us to do is run an A/B test with two conditions:
- Users who search for a term which redirects to an article get the
current experience (control) 2. Users who search for a term which redirects to an article get the article title in the search results claiming to be the redirect title (test)
I bet this would really improve the clickthrough rate for this class of searches. It would definitely improve the UX.
Sometimes I when I look for redirect, like "Obamma", I get this:
Barack Obama (redirect from Obamma)
Which seems to be the best way of presenting it, it both says what I looked for (even if it's wrong) and what I actually should have looked for (which is educational). Maybe it should show that always if it's redirect-driven?
Seems like it.
-- Stas Malyshev smalyshev@wikimedia.org
discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery