[Wikimedia-l] Monobook was optimised for editors, Vector is more balanced between readers and edtors

[Andrew Gray]

A/B testing major interface changes is very difficult. (I think we had
this same discussion over VisualEditor).

For example, in order to make the results comparable, you'll have to
update all the help pages & documentation to deal with both styles
("why doesn't my screen look like that?"), otherwise one becomes more
difficult to use & will have consequent drop-out rates. You also have
to try and deal with the fact that all readers (and indeed editors)
without accounts will use Vector, and will be immediately confused by
why it looks different once logged in - at which point many will get
frustrated.

A/B testing is good for small things like the login form, where it
works great, but won't always work for everything...

Andrew.

On 22 November 2013 07:24, Federico Leva (Nemo) <nemowiki at gmail.com> wrote:
> WereSpielChequers, 22/11/2013 08:03:
>
>> But it would be interesting to see some stats on the relative
>> retention and upgrading of editors who use monobook and Vector.
>
>
> The idea sounds crazy, but yes, why not, let's test this. I believe you can
> put your thoughts on a Meta-Wiki Research: page, describing the background,
> the A/B test and the proposed analysis, and then ask the WMF to run it
> (preferably with the consensus of the target wikis, but it's not usually
> considered necessary for so-called experimentations).
>
> Nemo
>
>
> _______________________________________________
> Wikimedia-l mailing list
> Wikimedia-l at lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:wikimedia-l-request at lists.wikimedia.org?subject=unsubscribe>



-- 
- Andrew Gray
  andrew.gray at dunelm.org.uk