On Mon, Jul 22, 2013 at 8:44 PM, Tim Starling <tstarling(a)wikimedia.org> wrote:
and the results from Aaron Halfaker's study [2]
As noted at the top of the page, the analysis is still in progress.
Importantly, there were many confounding variables in the test, some
of which are already documented. This includes users being assigned to
the test group that received VisualEditor whose browser did not
properly support it (it would have literally just not worked if the
user attempted to edit); these issues were fixed later. See
https://meta.wikimedia.org/wiki/Research:VisualEditor%27s_effect_on_newly_r…
for some of these issues, but like I said, analysis is still in
progress and we'll need to see what conclusions can actually be drawn
from the data.
A proponent of source editing would claim that the
steep learning
curve is justified by the end results. A visual editor is easier for
new users, but perhaps less convenient for power users. So Aaron
Halfaker's study took its measurements at the point in the learning
curve where you would expect the benefit of VE to be most clear: the
first edit.
Actually, as noted in the draft, because the test group was assigned
at the point of account creation, we're not taking into account any
prior experience using wikitext as an IP editor. 59% of respondents in
the 2011 editor survey stated that they had edited as IPs prior to
making an account, so we should assume that this is not an
insignificant proportion:
https://meta.wikimedia.org/wiki/Research:Wikipedia_Editors_Survey_November_…
Round-trip bugs
If you have, like I have, spent hours looking at VisualEditor diffs,
you'll know that these are relatively rare at this point. The bug
category of "round-trip bugs" is sometimes used for issues that aren't
accurately be described this way, e.g. users typing wikitext into
VisualEditor, having difficulty updating a template parameter, or
accidentally deleting content (sometimes due to bugs in VE).
Perhaps the main problem is performance. Perhaps new
users are
especially likely to quit on the first edit because they don't want to
wait 25-30 seconds for the interface to load (the time reported in
[3]). Performance is a very common complaint for established users also.
You're quoting a user test from June 10 which was performed on the
following page, which I've temporarily undeleted:
https://www.mediawiki.org/wiki/Golden-Crowned_Sparrow
Editing this page in Firefox on a 6-year-old system only slightly
faster than the tester's specs today takes about 5 seconds to
initialize. In Chrome it takes about 3 seconds, in the ballpark of
reloading the page into the source editor. Note that Gabriel put major
caching improvements into production around June 7, which may not have
been in effect for this user / this page yet.
Still, I think that the hypothesis that any actual negative impact of
VE on new users is due to performance issues is very supportable.
Performance is the single biggest issue overall for VE right now, and
performance on long pages can absolutely be prohibitively poor.
Improving it is the highest priority for the team.
Erik
--
Erik Möller
VP of Engineering and Product Development, Wikimedia Foundation