Hey everyone,
Last week I've done some usability testing with my InlineEditor extension, to see whether or not it's a good idea at all. I've still got to do formal analysis, but that'll take a while, so for now I'll only share with you the most important things I noticed while doing the tests.
First, a few remarks. The test group was heavily biased towards students of Information Sciences, so towards above average intelligence and familiarity with codes and programming. There were 3 test groups, testing both versions of the editor (one where you can select an edit mode like "Sentences", "References", "Media", etc., and one where you can select "Sentences", "Paragraphs", "Sections") and a control group.
1. The editing of sentences was great the new way. Users were much more comfortable seeing the article and being able to directly manipulate it. 2. Users did not really notice the other editing options. This is a huge problem, as the whole idea of this editor is to show the user an increasing amount of wikitext complexity. When users did notice it...: a. In the case of the functional edit options ("References", "Media", etc.), users used this more as a help center and then proceeded to the full (original) editor. b. In the case of the block edit options ("Paragraphs", "Sections"), users used it correctly and found it helpful. This might indicate that this is the right way to go (which was my hypothesis), but it still has to be approached differently. 3. Users did actually like the idea of the interface, whether or not they could accomplish their tasks. Users noticed the edit summary field earlier and used it more often (although the test groups were too small to draw a formal conclusion from this).
In conclusion: users really liked the direct manipulation, but did not get the different edit modes. So, back to the drawing board! ;)
Full interview videos will be available on Wikimedia Commons somewhere next month. They are in Dutch, though.
You can try the editor with block edit options at http://janpaulposma.nl/sle/wiki. It's still work in progress: do expect to run into bugs that I did not find worthy to fix yet. :)
Best regards, Jan Paul Posma