I was glad to see some WMF members speak their mind against the current stance from Erik and James. But when I see Erik answer, I'm clearly understanding that WMF management is simply being blind. Erik, if I read correctly your reply :
- You still don't have analysed the A/B test period : first evidence show that it has a negative feedback but you think further analysis could show otherwise. But, with first evidence pointing to negative feedback, you keep going on rolling out VE to as many people as possible without making sure before that this won't have more negative results. - "Performance is the single biggest issue for VE" : ouah... are you in denial ?
Nico
On Tue, Jul 23, 2013 at 7:23 AM, Erik Moeller erik@wikimedia.org wrote:
On Mon, Jul 22, 2013 at 8:44 PM, Tim Starling tstarling@wikimedia.org wrote:
and the results from Aaron Halfaker's study [2]
As noted at the top of the page, the analysis is still in progress.
Importantly, there were many confounding variables in the test, some of which are already documented. This includes users being assigned to the test group that received VisualEditor whose browser did not properly support it (it would have literally just not worked if the user attempted to edit); these issues were fixed later. See
https://meta.wikimedia.org/wiki/Research:VisualEditor%27s_effect_on_newly_re... for some of these issues, but like I said, analysis is still in progress and we'll need to see what conclusions can actually be drawn from the data.
A proponent of source editing would claim that the steep learning curve is justified by the end results. A visual editor is easier for new users, but perhaps less convenient for power users. So Aaron Halfaker's study took its measurements at the point in the learning curve where you would expect the benefit of VE to be most clear: the first edit.
Actually, as noted in the draft, because the test group was assigned at the point of account creation, we're not taking into account any prior experience using wikitext as an IP editor. 59% of respondents in the 2011 editor survey stated that they had edited as IPs prior to making an account, so we should assume that this is not an insignificant proportion:
https://meta.wikimedia.org/wiki/Research:Wikipedia_Editors_Survey_November_2...
Round-trip bugs
If you have, like I have, spent hours looking at VisualEditor diffs, you'll know that these are relatively rare at this point. The bug category of "round-trip bugs" is sometimes used for issues that aren't accurately be described this way, e.g. users typing wikitext into VisualEditor, having difficulty updating a template parameter, or accidentally deleting content (sometimes due to bugs in VE).
Perhaps the main problem is performance. Perhaps new users are especially likely to quit on the first edit because they don't want to wait 25-30 seconds for the interface to load (the time reported in [3]). Performance is a very common complaint for established users also.
You're quoting a user test from June 10 which was performed on the following page, which I've temporarily undeleted:
https://www.mediawiki.org/wiki/Golden-Crowned_Sparrow
Editing this page in Firefox on a 6-year-old system only slightly faster than the tester's specs today takes about 5 seconds to initialize. In Chrome it takes about 3 seconds, in the ballpark of reloading the page into the source editor. Note that Gabriel put major caching improvements into production around June 7, which may not have been in effect for this user / this page yet.
Still, I think that the hypothesis that any actual negative impact of VE on new users is due to performance issues is very supportable. Performance is the single biggest issue overall for VE right now, and performance on long pages can absolutely be prohibitively poor. Improving it is the highest priority for the team.
Erik
Erik Möller VP of Engineering and Product Development, Wikimedia Foundation
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l