Hello,
tl;dr - we’re running a second test for displaying sister projects in the search results (with new link highlights) on 8 wikis for a week; starting today, with apologies for sending to multiple email lists.
As a follow up to a recent A/B test https://phabricator.wikimedia.org/T149806 that the Discovery Search https://www.mediawiki.org/wiki/Wikimedia_Discovery/Search team conducted, we’ll be running a second test https://phabricator.wikimedia.org/T160004 for displaying sister project search results in a sidebar on several wiki’s search engine results page (SERP) - starting today. You can read the draft report https://commons.wikimedia.org/wiki/File:A_Test_Of_Cross-wiki_Search_-_Helping_Users_Discover_Content_On_Wikipedia%E2%80%99s_Sister_Projects.pdf or browse the next two paragraphs for the biggest concerns.
We realized that when we display the results from the sister projects, it might not have been intuitive to our visitors that there were links that could be clicked to go to individual articles and to additional search results, so, we’ve added blue links https://phabricator.wikimedia.org/T158935 to the results.
We also discovered that the smaller wikis in which we had originally chosen to test on, might not have had enough relevant results in their sister projects to be displayed. To remedy that, we reached out https://phabricator.wikimedia.org/T158935 to the community and added 4 larger wikis along with the original 4 wikis, to test on. The list is as follows: Persian, Italian, Catalan, Polish, Russian, French, German, and Arabic.
Here are a few links that will not affect our testing, but will demonstrate the new test display:
- cawiki: https://ca.wikipedia.org/wiki/Special:Search?search= manifestaci%C3%B3&fulltext=1&cirrusUserTesting=recall_ sidebar_results&searchToken=66xv5qgi97musdtybgwik11sk#manifestaci%C3%B3 https://ca.wikipedia.org/wiki/Special:Search?search=manifestaci%C3%B3&fulltext=1&cirrusUserTesting=recall_sidebar_results&searchToken=66xv5qgi97musdtybgwik11sk#manifestaci%C3%B3 - dewiki: https://de.wikipedia.org/wiki/Special:Search?search= regenbogen&fulltext=1&cirrusUserTesting=recall_ sidebar_results&searchToken=44wgqit3kykji2o1cxtaxlzcs#regenbogen https://de.wikipedia.org/wiki/Special:Search?search=regenbogen&fulltext=1&cirrusUserTesting=recall_sidebar_results&searchToken=44wgqit3kykji2o1cxtaxlzcs#regenbogen - arwiki: https://ar.wikipedia.org/wiki/Special:Search?search=%D9%82% D9%88%D8%B3+%D8%A7%D9%84%D9%85%D8%B7%D8%B1&fulltext=1& cirrusUserTesting=recall_sidebar_results&searchToken= bxfe1fnvj7sav1nue56hpjt69#%D9%82%D9%88%D8%B3%2B%D8%A7%D9%84% D9%85%D8%B7%D8%B1 https://ar.wikipedia.org/wiki/Special:Search?search=%D9%82%D9%88%D8%B3+%D8%A7%D9%84%D9%85%D8%B7%D8%B1&fulltext=1&cirrusUserTesting=recall_sidebar_results&searchToken=bxfe1fnvj7sav1nue56hpjt69#%D9%82%D9%88%D8%B3%2B%D8%A7%D9%84%D9%85%D8%B7%D8%B1
You can edit the url to change the search term and also change to a different Wikipedia. For example, try this url for “apple pie” on enwiki: https://en.wikipedia.org/wiki/Special:Search?search=apple+pie&fulltext=1... cirrusUserTesting=recall_sidebar_results&searchToken= 8mavijx36f9cl26l6k2v2o11l#apple%2Bpie
Let us know what you think!
Cheers from the Discovery Search Team -- deb tankersley irc: debt Product Manager, Discovery Wikimedia Foundation
Hi all,
I am glad this is live now -- I was offering my help with another sister projects interface back in December but unfortunately had to use the new year break for something else.
https://de.wikiquote.org/wiki/Special:Search?search=regenbogen&fulltext=...
A few thoughts regarding this:
0) I would suggest to notify everybody that they may report a new bug at https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?tags=discovery
1) I think the project icons are hard to distinguish. I would suggest to put a more colorful version which makes it easier for the readers to figure out what the relevant project is about without reading its name. Filed as https://phabricator.wikimedia.org/T160724
2) There is some empty space after the 'from X' line, so I would append a comma and a caption, i.e.
From Wikibooks, the open-content textbooks collection From Wikitravel, the FREE worldwide travel guide
Filed as https://phabricator.wikimedia.org/T160725
3) I would suggest a 'feedback' link next to the word 'Sister projects' word at the top of the column. I would expect it to open a textbox dialog immediately where the user can type their comment and click 'Submit' without any extra clicks. Filed as https://phabricator.wikimedia.org/T160726
4) I would suggest to add the count of search results for each sister project. In the "More results" link at the right bottom of each sister project rectangle it could say "More results (45) ..." to include the search results count. Filed as https://phabricator.wikimedia.org/T160727
Svetlana
Thanks for the development, tests and updates.
Compared to the traditional crosswiki search, this interface has some potential weaknesses. 1) Custom logos/icons are used, which have not been thoroughly tested in the past and are vastly inferior to the actual project logos (for instance they're black and white and the icon for the dictionary focuses on the Latin alphabet). A test is needed against the normal icons to see if there is a proven improvement: otherwies, just use the actual logos. 2) Showing a snippet means you're only able to show one search result from each wiki, if I see correctly. Cross-wiki search is most useful when one searches an ambiguous term which might yield unexpected results, so it takes quite a bit of optimism to assume that the first result will be effective. I suggest to test an alternative version where either some wiki gets more results shown than another (e.g. if the match score is close) or every wiki gets more results shown and there is no snippet. 3) The images take a lot of space. In many of my informal usability tests (looking over the people's shoulders while they complete wiki tasks at workshops), I often see people not finding the search results at all because the results are below the fold or just because stuff on top acts as a visual barrier. I suggest to try a variant without any multimedia results at all (there's already the multimedia tab anyway), or where they take much less space, so that the actual search results are as close as possible to a region where they're visible to the user.
Nemo
Nemo,
Do you think it would be useful to file all these as phab bugs? That's what I did but I see you did not. I would like to know what is better.
Svetlana
On 2017-03-17 18:31:50 +1100 "Federico Leva (Nemo)" nemowiki@gmail.com wrote:
Thanks for the development, tests and updates.
Compared to the traditional crosswiki search, this interface has some potential weaknesses.
- Custom logos/icons are used, which have not been thoroughly tested
in the past and are vastly inferior to the actual project logos (for instance they're black and white and the icon for the dictionary focuses on the Latin alphabet). A test is needed against the normal icons to see if there is a proven improvement: otherwies, just use the actual logos. 2) Showing a snippet means you're only able to show one search result from each wiki, if I see correctly. Cross-wiki search is most useful when one searches an ambiguous term which might yield unexpected results, so it takes quite a bit of optimism to assume that the first result will be effective. I suggest to test an alternative version where either some wiki gets more results shown than another (e.g. if the match score is close) or every wiki gets more results shown and there is no snippet. 3) The images take a lot of space. In many of my informal usability tests (looking over the people's shoulders while they complete wiki tasks at workshops), I often see people not finding the search results at all because the results are below the fold or just because stuff on top acts as a visual barrier. I suggest to try a variant without any multimedia results at all (there's already the multimedia tab anyway), or where they take much less space, so that the actual search results are as close as possible to a region where they're visible to the user.
Nemo
discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery
Svetlana Tkachenko, 17/03/2017 20:31:
Do you think it would be useful to file all these as phab bugs? That's what I did but I see you did not. I would like to know what is better.
Maybe the team can tell you what they prefer in this case. Mine are mostly suggestions/opinions/un-asked feedback for things to test in the next round if the team manages to, hence I sent them to the list for quick inspection.
Nemo
Hi Deb,
While I can't dive deeply into the implementation details at the moment, I just wanted to say thanks for working on cross-project search. I'd find it useful to be able to be on Wikipedia and be able to see related search results from Wiktionary, Commons, Wikisource, etc. So this an expression of "support in principle". (:
Pine
On Thu, Mar 16, 2017 at 6:56 PM, Deborah Tankersley < dtankersley@wikimedia.org> wrote:
Hello,
tl;dr - we’re running a second test for displaying sister projects in the search results (with new link highlights) on 8 wikis for a week; starting today, with apologies for sending to multiple email lists.
As a follow up to a recent A/B test https://phabricator.wikimedia.org/T149806 that the Discovery Search https://www.mediawiki.org/wiki/Wikimedia_Discovery/Search team conducted, we’ll be running a second test https://phabricator.wikimedia.org/T160004 for displaying sister project search results in a sidebar on several wiki’s search engine results page (SERP) - starting today. You can read the draft report https://commons.wikimedia.org/wiki/File:A_Test_Of_Cross-wiki_Search_-_Helping_Users_Discover_Content_On_Wikipedia%E2%80%99s_Sister_Projects.pdf or browse the next two paragraphs for the biggest concerns.
We realized that when we display the results from the sister projects, it might not have been intuitive to our visitors that there were links that could be clicked to go to individual articles and to additional search results, so, we’ve added blue links https://phabricator.wikimedia.org/T158935 to the results.
We also discovered that the smaller wikis in which we had originally chosen to test on, might not have had enough relevant results in their sister projects to be displayed. To remedy that, we reached out https://phabricator.wikimedia.org/T158935 to the community and added 4 larger wikis along with the original 4 wikis, to test on. The list is as follows: Persian, Italian, Catalan, Polish, Russian, French, German, and Arabic.
Here are a few links that will not affect our testing, but will demonstrate the new test display:
search=manifestaci%C3%B3&fulltext=1&cirrusUserTesting= recall_sidebar_results&searchToken=66xv5qgi97musdtybg wik11sk#manifestaci%C3%B3 https://ca.wikipedia.org/wiki/Special:Search?search=manifestaci%C3%B3&fulltext=1&cirrusUserTesting=recall_sidebar_results&searchToken=66xv5qgi97musdtybgwik11sk#manifestaci%C3%B3
search=regenbogen&fulltext=1&cirrusUserTesting=recall_sideba r_results&searchToken=44wgqit3kykji2o1cxtaxlzcs#regenbogen https://de.wikipedia.org/wiki/Special:Search?search=regenbogen&fulltext=1&cirrusUserTesting=recall_sidebar_results&searchToken=44wgqit3kykji2o1cxtaxlzcs#regenbogen
search=%D9%82%D9%88%D8%B3+%D8%A7%D9%84%D9%85%D8%B7%D8%B1& fulltext=1&cirrusUserTesting=recall_sidebar_results& searchToken=bxfe1fnvj7sav1nue56hpjt69#%D9%82%D9%88%D8%B3%2B% D8%A7%D9%84%D9%85%D8%B7%D8%B1 https://ar.wikipedia.org/wiki/Special:Search?search=%D9%82%D9%88%D8%B3+%D8%A7%D9%84%D9%85%D8%B7%D8%B1&fulltext=1&cirrusUserTesting=recall_sidebar_results&searchToken=bxfe1fnvj7sav1nue56hpjt69#%D9%82%D9%88%D8%B3%2B%D8%A7%D9%84%D9%85%D8%B7%D8%B1
You can edit the url to change the search term and also change to a different Wikipedia. For example, try this url for “apple pie” on enwiki: https://en.wikipedia.org/wiki/Special:Search? search=apple+pie&fulltext=1&cirrusUserTesting=recall_sidebar _results&searchToken=8mavijx36f9cl26l6k2v2o11l#apple%2Bpie
Let us know what you think!
Cheers from the Discovery Search Team
deb tankersley irc: debt Product Manager, Discovery Wikimedia Foundation
discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery