On Tue, Apr 15, 2014 at 9:59 AM, Gergo Tisza gtisza@wikimedia.org wrote:
- instead of guessing about user preferences, you could just create a
simple survey which shows them the same text with two different font stacks side by side, and ask them which is more readable. This is good for making aesthetic decisions more objective, and also for catching weird issues with old machines, CJK fonts etc: you can add a comment field to the survey, and if the browser is sufficiently modern to support canvas elements, you can even save a snapshot if the rendered text; you can skim through the survey replies which are different from what you have expected, and look for display problems.
Are you volunteering to build such a survey tool? ;-)
We don't have a powerful/easy to use/not annoying/privacy-respecting survey tool that can do side-by-side comparisons. This is why the feature was launched using Beta Features for five months first. Putting out in opt-in mode and gathering feedback via the channels we have now is the most efficient way to make a change that doesn't have a big WMF team assigned to like Multimedia or VisualEditor.
When it comes to using a survey to catch problems early and gauging preferences, a survey still very much suffers from the self-selection bias that all opt-in options have. It's just the name of the game. When you move something from opt-in to opt-out you reach a wider audience and encounter new complaints/questions/bugs.