New subject: WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)

2 May 2011

On Mon, May 2, 2011 at 12:04 AM, Tim Starling &lt;tstarling(a)wikimedia.org&gt;wrote;wrote:

...
  Can someone please tell me, in precise technical
terms, what is wrong
 with Wikia's WYSIWYG editor and why we can't use it?

 I have heard that it has bugs in it, but I have not been told exactly
 what these bugs are, why they are more relevant for Wikimedia than for
 Wikia, or why they can't be fixed.

 Years ago, we talked dismissively about WYSIWYG. We discussed the
 features that a WYSIWYG editor would have to have, pointing out how
 difficult they would be to implement and how we didn't have the
 manpower to pull off such a thing. Now that Wikia has gone ahead and
 implemented those exact features, what is the problem?

The most fundamental problem with Wikia's editor remains its fallback
behavior when some structure is unsupported:

  "Source mode required

  Rich text editing has been disabled because the page contains complex
code."

Here's an example of unsupported code, the presence of which makes a page
permanently uneditable by the rich editor until it's removed:

  <table>
  <tr><td>a</td></tr>
  </table>

You can try this out now at http://communitytest.wikia.com/

It will at least let you edit other *sections* that don't contain anything
that scares it, but if the nasty bit is somewhere in what you want to edit,
it just doesn't recover.

There are some smart things in what they're doing: annotating the markup
ought to be a big help in hooking up the rendered HTML bits back to the
original source. The way they hold template invocations and plugins as
standalone placeholders within the rich text is pretty good (and could be a
bit better if it could display some content and provide even more advanced
invocation editing tools, which is all detail work).

But if it just gives up on entire pages, we've got a problem because to
handle Wikipedia we need to handle lllooonnnggg pages that tend to include
lots of complex templates which pull in funky code of their own.

At a minimum, assuming that other round-tripping problems are all resolved
and the treatment of templates and extensions can be improved, it would need
to be changed to recognize uneditable chunks and present them as a sort of
placeholders too -- like the templates you should be able to dive into
source and edit them if need be, but they ought not destroy the rest of the
page.

Beyond that let's flip the question the other way -- what do we *want* out
of WYSIWYG editing, and can that tool provide it or what else do we need?
I've written up some notes a few weeks ago, which need some more collation &
updating from the preliminary experiments I'm doing, and I would strongly
appreciate more feedback from you Tim and from everyone else who's been
poking about in parser & editing land:

  http://www.mediawiki.org/wiki/Wikitext.next

And also some of Trevor's notes which I have poked at:

  http://www.mediawiki.org/wiki/Visual_Editor_design

I've got some aggressive ideas about normalizing how we deal with template
expansion to work at the parse tree level; this can be friendlier to some
levels of caching, splitting portions of parsing between PHP and optimized
native code, or even mixing some things between pre-parsed text and
client-side work, but most importantly I'm interested in making sure we have
a relatively clean hierarchical relationship between parts of the document,
which we can use to much more reliably hook up parts of the rendered HTML
output:

* maintain an abstract parse tree that can be hooked up fully to both the
original source text *and* the live output DOM
* do section, paragraph, or table-cell editing inline directly on a view
page, with predictable replacements

It may well be that this is too expansive and we'll want to contract to
something that's more like Wikia's annotated parser output -- in most cases
it should give us similar information but it'll probably be harder to
replace parts of the page at runtime in JavaScript.

Another goal beyond editing itself is normalizing the world of 'alternate
parsers'. There've been several announced recently, and we've got such a
large array now of them available, all a little different. We even use mwlib
ourselves in the PDF/ODF export deployment, and while we don't maintain that
engine we need to coordinate a little with the people who do so that new
extensions and structures get handled.

A new visual editor that's built around a normalized, defined parser could
be a great help; other folks will be able to use compatible parsers instead
of mostly-similar parsers.

For the moment I'm mostly schooling myself on the current state of the world
and setting up experimental tools to aid in debugging extra
parser/editor-related goodies (eg the inspector tool I'm fiddling with at
http://en.wikipedia.org/wiki/User:Brion_VIBBER/vector.js ), but hope to get
some of these projects starting moving forward after Berlin.

-- brion