Visual Editor and Parsoid New Pages in Wikitext?

List overview All Threads
Download

newer

older

The place of language support in...

GSoC 2014 Project

Derric Atzrott

14 Feb 2014 14 Feb '14

7:32 p.m.

If I install the Visual Editor mediawiki extension on the Wiki that I manage here at my work and also setup Parsoid, will new pages be saved with Wikitext still? The documentation for the extension's installation mentions that after you install Visual Editor you won't be able to edit Wikitext pages, but you will be able to create new pages. It also mentions that with Parsoid you will be able to edit Wikitext pages again. This leaves unclear whether or not having Parsoid installed makes Visual Editor save new pages as Wikitext as well, or just previously existing pages. I'm thinking about letting folks try Visual Editor here, but I need to be able to work with Wikitext still as there are definitely some unusual things I do in Wikitext from time to time. Thank you, Derric Atzrott Computer Specialist Alizee Pathology

Show replies by date

Gabriel Wicke

14 Feb 14 Feb

8:35 p.m.

On 02/14/2014 11:32 AM, Derric Atzrott wrote:

...

If I install the Visual Editor mediawiki extension on the Wiki that I manage here at my work and also setup Parsoid, will new pages be saved with Wikitext still?

Yes. All content is currently stored as wikitext, so you are naturally able to use the wikitext editor to create pages or edit existing pages. In the longer term we are working on the ability to store HTML instead for new wikis, in which case it might become possible to run without Parsoid if you don't need a wikitext editor front-end. This is not going to happen over night and will just be an option, so no reason to worry.

...

The documentation for the extension's installation mentions

Do you have a link to this documentation? Thanks, Gabriel

Derric Atzrott

8:47 p.m.

...

If I install the Visual Editor mediawiki extension on the Wiki that I manage here at my work and also setup Parsoid, will new pages be saved with Wikitext still?

So right now, assuming I don't have Parsoid, Visual Editor creates pages in Wikitext, it just can't edit previously existing pages? I would assume that this also means that without Parsoid if I edit a page in the regular Wikitext editor, I won't be able to use Visual Editor with it anymore.

...

The documentation for the extension's installation mentions

Do you have a link to this documentation?

I was reading this page: https://www.mediawiki.org/wiki/Extension:VisualEditor Thank you, Derric Atzrott

Gabriel Wicke

9:39 p.m.

On 02/14/2014 12:47 PM, Derric Atzrott wrote:

...

If I install the Visual Editor mediawiki extension on the Wiki that I manage here at my work and also setup Parsoid, will new pages be saved with Wikitext still?

VisualEditor is an HTML editor and doesn't know about wikitext. All conversions between wikitext and HTML are done by Parsoid. You need Parsoid if you want to use VisualEditor on current wikis. Gabriel

Lars Aronsson

11:40 p.m.

On 02/14/2014 10:39 PM, Gabriel Wicke wrote:

...

VisualEditor is an HTML editor and doesn't know about wikitext.

That single sentence explains so much. If you would have told me two years ago, I could have said "wrong path, this will fail". (Of course, many would have protested and refused to listen to sound advice, but now we know the outcome.) What do you predict we will be using five years from now, in 2019? Plain old wikitext, VisualEditor, or some other path? -- Lars Aronsson (lars(a)aronsson.se) Aronsson Datateknik - http://aronsson.se

James Forrester

15 Feb 15 Feb

12:55 a.m.

On 14 February 2014 15:40, Lars Aronsson <lars(a)aronsson.se> wrote:

...

On 02/14/2014 10:39 PM, Gabriel Wicke wrote:

VisualEditor is an HTML editor and doesn't know about wikitext.

Thanks for your timely input. :-) We pointed out some of the (dozens of) reasons why a client-side implementation of the parser wasn't a workable solution when we made the choice to take this route; sorry you didn't notice that at the time. What do you predict we will be using five

...

years from now, in 2019? Plain old wikitext, VisualEditor, or some other path?

For new out-of-the-box MediaWiki installs? Almost certainly HTML. For existing legacy MediaWiki sites with sane quantities of extensions? Probably HTML for most. For WMF's currently-hosted sites? Probably wikitext, but it's possible that we'll have switched. J. -- James D. Forrester Product Manager, VisualEditor Wikimedia Foundation, Inc. jforrester(a)wikimedia.org | @jdforrester

Matthew Flaschen

8:06 a.m.

On 02/14/2014 06:40 PM, Lars Aronsson wrote:

...

On 02/14/2014 10:39 PM, Gabriel Wicke wrote:

VisualEditor is an HTML editor and doesn't know about wikitext.

That single sentence explains so much.

I think Gabriel is simplifying a little. VisualEditor is an editor that understands, edits, and preserves semantically rich, augmented, HTML 5. If you look at the HTML source code of http://parsoid.wmflabs.org/enwiki/Earth , you can see that while it is HTML, it is carefully augmented with additional information. Among other things, that allows VisualEditor to handle MediaWiki-specific features, such as templates. After the user is done editing with VisualEditor, Parsoid then turns that augmented HTML back into wikitext, generally without dirty diffs.

...

If you would have told me two years ago, I could have said "wrong path, this will fail". (Of course, many would have protested and refused to listen to sound advice, but now we know the outcome.)

If you're saying that VisualEditor has failed, I disagree. I don't judge software quality by whether it's opt-in or opt-out (this is a question of configuration, and will probably change over time), but on its own merits. VisualEditor has already fulfilled many requirements, and is still improving. Matt Flaschen

David Gerard

16 Feb 16 Feb

9:34 a.m.

On 15 February 2014 08:06, Matthew Flaschen <mflaschen(a)wikimedia.org> wrote:

...

If you look at the HTML source code of http://parsoid.wmflabs.org/enwiki/Earth , you can see that while it is HTML, it is carefully augmented with additional information.

Currently giving "Error: EROFS, read-only file system" :-) - d.

addshorewiki

12:59 p.m.

That is due to Wikimedia Labs currently having a small NFS issue. Addshore On 16 February 2014 10:34, David Gerard <dgerard(a)gmail.com> wrote:

...

On 15 February 2014 08:06, Matthew Flaschen <mflaschen(a)wikimedia.org> wrote:

If you look at the HTML source code of http://parsoid.wmflabs.org/enwiki/Earth , you can see that while it is

HTML,

it is carefully augmented with additional information.

Currently giving "Error: EROFS, read-only file system" :-) - d. _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

David Gerard

9:34 a.m.

On 14 February 2014 23:40, Lars Aronsson <lars(a)aronsson.se> wrote:

...

What do you predict we will be using five years from now, in 2019? Plain old wikitext, VisualEditor, or some other path?

I would hope Wikitext had been stabbed through its black and twisted little heart, or at least as much as one can reasonably do that with ~5 billion words of legacy content. Horrible, horrible thing. - d.

Daniel Kinzler

15 Feb 15 Feb

8:05 p.m.

Am 14.02.2014 22:39, schrieb Gabriel Wicke:

...

VisualEditor is an HTML editor and doesn't know about wikitext. All conversions between wikitext and HTML are done by Parsoid. You need Parsoid if you want to use VisualEditor on current wikis.

Implementing a HTML content type in mediawiki would be pretty trivial. That way, a page could "natively" contain HTML, with no need of conversion. Anyone up to doing it?... -- daniel

Gabriel Wicke

16 Feb 16 Feb

4:49 a.m.

On February 15, 2014 12:05:49 PM PST, Daniel Kinzler <daniel(a)brightbyte.de> wrote:

...

Implementing a HTML content type in mediawiki would be pretty trivial. That way, a page could "natively" contain HTML, with no need of conversion. Anyone up to doing it?...

We are working towards this, but actual HTML-only MediaWiki is not as trivial as you make it sound. In the Parsoid roadmap we are discussing some of the issues involved. We might be able to use Parsoid HTML for page views (for both anonymous and logged-in views) before we have proper support for HTML-only wikis. Gabriel

David Gerard

9:32 a.m.

On 15 February 2014 20:05, Daniel Kinzler <daniel(a)brightbyte.de> wrote:

...

Am 14.02.2014 22:39, schrieb Gabriel Wicke:

...

> VisualEditor is an HTML editor and doesn't know about wikitext. All > conversions between wikitext and HTML are done by Parsoid. You need > Parsoid if you want to use VisualEditor on current wikis.

...

Implementing a HTML content type in mediawiki would be pretty trivial. That way, a page could "natively" contain HTML, with no need of conversion. Anyone up to doing it?...

There are extensions that allow raw HTML widgets, just putting them through unchecked. The hard part will be checking. Note that the rawness of the somewhat-filtered HTML is a part of WordPress's not so great security story (though they've had a lot less "update now!" in the past year). So, may not involve much less parsing. - d.

Gabriel Wicke

4:30 p.m.

On 02/16/2014 01:32 AM, David Gerard wrote:

...

The difference is that you can run the sanitizer on save, and then only need to re-run it when a bug in it was fixed (which can happen in a background job rather than on view). We will maintain a sanitization level in storage to track the degree to which the HTML is sanitized. Sanitization is also the last part of parsing from wikitext to HTML. It is one of the cheapest parts of the parsing process, so running just that on a DOM is much cheaper than parsing from scratch. Gabriel

Daniel Kinzler

17 Feb 17 Feb

12:45 p.m.

Am 16.02.2014 10:32, schrieb David Gerard:

...

There are extensions that allow raw HTML widgets, just putting them through unchecked.

I know, I wrote one :) But that's not the point. The point is maintaining editable content as HTML instead of Wikitext.

...

The hard part will be checking.

Wikitext already allows a wide range of HTML tags, and we have a pretty good sanitizer for that. Adding support for a few additional structures (like links and images) and the extra data embedded by/for parsoid should not be a lot of work.

...

Note that the rawness of the somewhat-filtered HTML is a part of WordPress's not so great security story (though they've had a lot less "update now!" in the past year). So, may not involve much less parsing.

I think it would, since it doesn't add much to the sanitizer we use now, but reducing the amount of parsing wasn't the point. The point was avoiding conversion, which is potentially lossy and confusing, and essentially pointless. If we edit using an HTML ewditor, why not store HTML, make (structural) HTML diffs, etc? It just seems a lot more streight forward. -- daniel

Derric Atzrott

12:32 p.m.

...

In the longer term we are working on the ability to store HTML instead for new wikis, in which case it might become possible to run without Parsoid if you don't need a wikitext editor front-end. This is not going to happen over night and will just be an option, so no reason to worry.

VisualEditor is an HTML editor and doesn't know about wikitext. All conversions between wikitext and HTML are done by Parsoid. You need Parsoid if you want to use VisualEditor on current wikis.

This is what I had originally thought, but then somehow along the way I managed to get confused. Thank you for clarifying. Thank you, Derric Atzrott

3718

days inactive

3721

days old

wikitech-l@lists.wikimedia.org

Manage subscription

15 comments

8 participants

tags (0)

participants (8)

addshorewiki
Daniel Kinzler
David Gerard
Derric Atzrott
Gabriel Wicke
James Forrester
Lars Aronsson
Matthew Flaschen