I noticed that clicks to the hamburger menu icon in the top left corner and the languages icon at the bottom of the page were very similar [1], so I thus thought it would be interested to look at both of these elements on a per project level.
I took a sample day and constructed this table comparing clicks to the language button compared to the languages button.
I then divide clicks to hamburger by clicks to languages.
My hypothesis is that if clicks to the language icon are considerably higher than clicks to the hamburger icon the less recognisable the hamburger icon is in that language.
I filtered out wikis where the clicks to languages were less than 50, as I decided the data set for those were too small.
Interestingly for enwiki the score is close to 1 (0.9625941071), a language I would expect this icon to translate well.
For azwiki (Azerbaijani language) the language button has considerably more clicks - 1366 compared to 487 (0.3565153734)
Other WIkipedias where the hamburger icon might not be translating well (where score is less than 0.5): Bosnian, Polish, Japanese, Korean.
I've shared my data on a public URL, feel free to explore, analyse, comment [1]. As a next step it would be interesting to pick a project e.g. Japanese, monitor clicks to hamburger vs language and see how these values change with a different value.
[1] http://mobile-reportcard.wmflabs.org/#graph-limn225 ui daily graph [2] https://docs.google.com/spreadsheets/d/1aCddwCiTCrDdKivLIYFujef69k-uB0DtsqJq...
Jon Robson, 04/12/2014 20:58:
My hypothesis is that if clicks to the language icon are considerably higher than clicks to the hamburger icon the less recognisable the hamburger icon is in that language.
Interesting idea and nice way to get some data. However, I could speculate that if a wiki has many pages with language links, especially stubs, users are more likely to use them and read the pages in other languages. If pages satisfy the users, or don't have language links, or don't exist at all, then it's harder to get language clicks links (users will reach other languages in other ways, e.g. search engines).
Can the count be restricted to pages where language links are available?
Nemo
The similarity in the numbers is indeed striking, but I don't think that it says much about the perception of the hamburger icon. I suspect that for most readers the language button is more useful than *any* of the actions in the hamburger menu - home, random, nearby, watchlist, settings, log in. To confirm it, I'd love to see the the numbers for these other actions.
Also, as Nemo asks, it would be useful to see pages without language links separated, and to also see page length taken into account somehow - on a short page it is easier to see the languages button (less or no scrolling), and there's more motivation to tap it (the hope to read more in another language).
You can see this in comparison to other features.
Yes it is indeed more useful than items in the menu but as you can see in the graph the features are near identical in terms of clicks tracked. However what striked me as odd was that the number of clicks varied drastically depending on the language (although correlated for all languages they are not collerated per language).
Yes this could mean other things such as languages are more important in the given language and thus it is more useful (maybe the language is incomplete) and tied to what you and Nemo suggested it is more prevalent on the screen. Yet i cant help but wonder if it also might hint at something to do with the icons effectiveness especially when I look at well developed wiki's such as Chinese and Japanese when compared to English.
By the way, we don't have that much fine grained information with regards to shat type of pages they clicked these features on.
It might be useful to try a different logo on one of the projects e.g. Japanese and see how this gets impacted by the change. If there is no drastic change we could probably conclude that indeed my comparison sucks :) We can certainly use clicks as a guideline of whether the icon is getting more effective.
I'm ccing analytics in case they have any views on this. On Dec 5, 2014 2:26 AM, "Amir E. Aharoni" amir.aharoni@mail.huji.ac.il wrote:
The similarity in the numbers is indeed striking, but I don't think that it says much about the perception of the hamburger icon. I suspect that for most readers the language button is more useful than *any* of the actions in the hamburger menu - home, random, nearby, watchlist, settings, log in. To confirm it, I'd love to see the the numbers for these other actions.
Also, as Nemo asks, it would be useful to see pages without language links separated, and to also see page length taken into account somehow - on a short page it is easier to see the languages button (less or no scrolling), and there's more motivation to tap it (the hope to read more in another language).
Jon Robson, 05/12/2014 17:56:
It might be useful to try a different logo on one of the projects e.g. Japanese and see how this gets impacted by the change. If there is no drastic change we could probably conclude that indeed my comparison sucks :) We can certainly use clicks as a guideline of whether the icon is getting more effective.
I think you should probably also consider whether a click on the icon is followed by a click on a menu item, to exclude most unintentional clicks. But yes, I agree it would be very interesting to try out some alternatives and see those numbers.
Nemo