MediaWiki 2.0 (was: No more Architecture Committee?)

List overview All Threads
Download

newer

older

Brion's role change within WMF

Getting the full URL of an image

James Forrester

16 Jan 2015 16 Jan '15

5:04 p.m.

[Moving threads for on-topic-ness.] On 16 January 2015 at 07:01, Brian Wolff <bawolff(a)gmail.com> wrote:

...

Does anyone actually have anything they want that is difficult to do currently and requires a mass compat break?

Sure. Three quick examples of things on the horizon (I'm not particularly saying we'd actually do these for Wikimedia's use, but if you're going to ask for straw man arguments… :-)): - Get rid of wikitext on the server-side. - HTML storage only. Remove MWParser from the codebase. All extensions that hook into wikitext (so, almost all of them?) will need to be re-written. - Real-time collaborative editing. - Huge semantic change to the concept of a 'revision'; we'd probably need to re-structure the table from scratch. Breaking change for many tools in core and elsewhere. - Replace local PHP hooks with proper services interfaces instead. - Loads of opportunities for improvements here (anti-spam tools 'as a service', Wordpress style; pre-flighting saves; ), but again, pretty much everything will need re-writing; this would likely be "progressive", happening one at a time to areas where it's useful/wanted/needed, but it's still a huge breaking change for many extensions.

...

Proposing to rewrite mediawiki because we can without even a notion of what we would want to do differently seems silly.

Show replies by date

Dmitriy Sintsov

16 Jan 16 Jan

7:22 p.m.

New subject: MediaWiki 2.0 (was: No more Architecture Committee?)

Why wikitext is so much disliked? It's more compact to type than HTML. It's a templating language. HTML is not. Then something like Handlebars (which is weaker than wikitext) should be used. Or, something like web components and custom tags. But why removing nice thing (wikitext) which saves a lot of keystrokes to break the whole codebase? Strange. Especially because there's more progressive Parsoid. It looks like whole MW can be rewritten in Node.js, not Python. On Fri, Jan 16, 2015 at 8:04 PM, James Forrester <jforrester(a)wikimedia.org> wrote:

...

[Moving threads for on-topic-ness.] On 16 January 2015 at 07:01, Brian Wolff <bawolff(a)gmail.com> wrote:

Does anyone actually have anything they want that is difficult to do currently and requires a mass compat break?

Proposing to rewrite mediawiki because we can without even a notion of what we would want to do differently seems silly.

Oh, absolutely. I think RobLa's point was that it's unclear who feels empowered to make that decision (rather than the pitch). I don't. I don't think RobLa does. Clearly the Architecture Committee don't. J. -- James D. Forrester Product Manager, Editing Wikimedia Foundation, Inc. jforrester(a)wikimedia.org | @jdforrester _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Brian Wolff

17 Jan 17 Jan

11:11 a.m.

New subject: MediaWiki 2.0 (was: No more Architecture Committee?)

On Jan 16, 2015 1:05 PM, "James Forrester" <jforrester(a)wikimedia.org> wrote:

...

[Moving threads for on-topic-ness.] On 16 January 2015 at 07:01, Brian Wolff <bawolff(a)gmail.com> wrote:

Does anyone actually have anything they want that is difficult to do currently and requires a mass compat break?

Sure. Three quick examples of things on the horizon (I'm not particularly

saying

...

we'd actually do these for Wikimedia's use, but if you're going to ask for straw man arguments… :-)): - Get rid of wikitext on the server-side. - HTML storage only. Remove MWParser from the codebase. All extensions that hook into wikitext (so, almost all of them?) will

need to

...

be re-written. - Real-time collaborative editing. - Huge semantic change to the concept of a 'revision'; we'd probably need to re-structure the table from scratch. Breaking change for many tools in core and elsewhere. - Replace local PHP hooks with proper services interfaces instead. - Loads of opportunities for improvements here (anti-spam tools 'as a service', Wordpress style; pre-flighting saves; ), but again,

pretty much

...

everything will need re-writing; this would likely be "progressive", happening one at a time to areas where it's useful/wanted/needed, but it's still a huge breaking change for many extensions.

Woo strawmen for me to shoot down! :) Actually, The revision thing is fair. Its pretty engrained that a pages have linear revisions, and each one has a single author, allowing multiple authors or branching and remerging would probably be a big enough change to warrant calling it 2.0. And it kind of fits, since last time structure of revisions was really rearranged afaik was 1.5. Without going into the merits/drawbacks of html storage - i dont see that being rewrite worthy. Whether the blob of data in the text table/es is html or wikitext doesnt really matter to mw. Especially with content handler. Hooks are a very active area of code. The sort of area where i would guess that adding an extra 20ms of time per each hook invocation to call an external service would not be ok (since there are hundreds of hook calls in a request). I dont think the notion of a service fits with how hooks are used, at least for many hooks. Of course someone could make a heavier weight version of hooks for where its a good idea. Or even just a wrapper object that forwards things to a service. I dont think this is worthy of a 2.0. --bawolff

Jay Ashworth

19 Jan 19 Jan

1:29 a.m.

New subject: I haven't looked lately...

----- Original Message -----

...

From: "James Forrester" <jforrester(a)wikimedia.org>

...

Three quick examples of things on the horizon (I'm not particularly saying we'd actually do these for Wikimedia's use, but if you're going to ask for straw man arguments… :-)): - Get rid of wikitext on the server-side.

Are we still litigating "should we piss off the power user editors* in the service of making things easier for notional new ones"? Or have I misunderstood the suggestion? Cheers, -- jra [* understanding that that's not the *only* thing which killing off wikitext breaks, merely the one that came first to mind ] -- Jay R. Ashworth Baylink jra(a)baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274

Daniel Friesen

1:41 a.m.

New subject: I haven't looked lately...

On 2015-01-18 5:29 PM, Jay Ashworth wrote:

...

----- Original Message -----

From: "James Forrester" <jforrester(a)wikimedia.org> Three quick examples of things on the horizon (I'm not particularly saying we'd actually do these for Wikimedia's use, but if you're going to ask for straw man arguments… :-)): - Get rid of wikitext on the server-side.

Parsoid can do Parsoid DOM to WikiText conversions. So I believe the suggestion is that storage be switched entirely to the Parsoid DOM and WikiText in classic editing just becomes a method of editing the content that is stored as Parsoid DOM in the backend. ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]

Jay Ashworth

1:46 a.m.

New subject: I haven't looked lately...

----- Original Message -----

...

From: "Daniel Friesen" <daniel(a)nadir-seen-fire.com>

...

On 2015-01-18 5:29 PM, Jay Ashworth wrote: > ----- Original Message ----- >> From: "James Forrester" <jforrester(a)wikimedia.org> >> Three quick examples of things on the horizon (I'm not >> particularly saying >> we'd actually do these for Wikimedia's use, but if you're going to >> ask for straw man arguments… :-)): >> >> - Get rid of wikitext on the server-side. > Are we still litigating "should we piss off the power user editors* > in the > service of making things easier for notional new ones"? > > Or have I misunderstood the suggestion?

...

Aha! Sorry; I was behind on Parsoid, was the problem. Yeah, if there's a way to edit in MWtext, both for humans and programs, then that serves the use cases I would be concerned about. Thanks for the prompt clarification, Daniel. Cheers, -- jra -- Jay R. Ashworth Baylink jra(a)baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274

David Gerard

1:43 p.m.

New subject: I haven't looked lately...

On 19 January 2015 at 01:46, Jay Ashworth <jra(a)baylink.com> wrote:

...

There's also fourteen years of legacy page versions which I think absolutely nobody is going to convert. - d.

James Forrester

20 Jan 20 Jan

5:11 p.m.

New subject: I haven't looked lately...

On 19 January 2015 at 05:43, David Gerard <dgerard(a)gmail.com> wrote:

...

On 19 January 2015 at 01:46, Jay Ashworth <jra(a)baylink.com> wrote:

Aha! Sorry; I was behind on Parsoid, was the problem. Yeah, if there's a way to edit in MWtext, both for humans and programs, then that serves

the

use cases I would be concerned about. Thanks for the prompt

clarification,

Daniel.

There's also fourteen years of legacy page versions which I think absolutely nobody is going to convert.

Code is very good at "just doing it" if we wanted to do it. A few billion revisions are just ephemeral details if it actually makes sense to change over. Not that we should do it blindly, but this shouldn't hold us back. J. -- James D. Forrester Product Manager, Editing Wikimedia Foundation, Inc. jforrester(a)wikimedia.org | @jdforrester

Federico Leva (Nemo)

6:34 p.m.

New subject: I haven't looked lately...

Really, James? WMF still has to cleanup the conversion from UseModWiki: https://phabricator.wikimedia.org/T2323 When bug 323 and others are fixed, I may believe you about billions-revisions conversions. Nemo

Jeroen De Dauw

8:04 p.m.

New subject: MediaWiki 2.0 (was: No more Architecture Committee?)

Hey, FYI: we had this discussion for SMW some time ago. We where at version 1.9, following a naming pattern very similar to the MediaWiki one. The question there was if the next release containing a break should be either 1.10 (no change), 2.0 (following semver) or 10.0 (dropping the "1."). People generally preferred 2.0, mostly because "10.0 seems like such a big jump". So this is what we went with. And while the 2.0 release was not more significant than 1.9 or 1.8, no one seems to have gotten seriously confused about it. - Get rid of wikitext on the server-side.

...

- HTML storage only. Remove MWParser from the codebase. All extensions that hook into wikitext (so, almost all of them?) will need to be re-written.

Just to confirm: this is not actually on the WMF roadmap right? :) Cheers -- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate Evil software architect at Wikimedia Germany ~=[,,_,,]:3

James Forrester

8:21 p.m.

New subject: MediaWiki 2.0 (was: No more Architecture Committee?)

On 20 January 2015 at 12:04, Jeroen De Dauw <jeroendedauw(a)gmail.com> wrote:

...

> - Get rid of wikitext on the server-side.

- HTML storage only. Remove MWParser from the codebase. All extensions that hook into wikitext (so, almost all of them?) will need to be re-written.

Just to confirm: this is not actually on the WMF roadmap right? :)

It's certainly not what I'm working on for the next year or so. It is unlikely to be something we do for WMF usage; it's more valuable to people that want VisualEditor but want PHP-only (or don't want Node) for the server. J. -- James D. Forrester Product Manager, Editing Wikimedia Foundation, Inc. jforrester(a)wikimedia.org | @jdforrester

Brian Wolff

9:53 p.m.

New subject: MediaWiki 2.0 (was: No more Architecture Committee?)

On Jan 20, 2015 4:22 PM, "James Forrester" <jforrester(a)wikimedia.org> wrote:

...

On 20 January 2015 at 12:04, Jeroen De Dauw <jeroendedauw(a)gmail.com>

wrote:

...

> > > - Get rid of wikitext on the server-side. > > - HTML storage only. Remove MWParser from the codebase. All > > extensions that hook into wikitext (so, almost all of them?)

will

...

need to be re-written.

Just to confirm: this is not actually on the WMF roadmap right? :)

Hypothetically how would this work? Wouldnt you still need parsoid to verify the html corresponds to some legit wikitext? Otherwise how would you know its safe? Since we are somewhat having a discussion about this (i recognize that this isnt a "real" discussion in the sense that there isnt a full technical proposal, or any concrete plans to actually do it in near future, just a wild idea that some people like), one of the reasons im somewhat skeptical about this idea, is if there is an xss issue, you seem much more screwed with html storage, since now the bad stuff is in the canonical representation, instead of wikitext storage where you can just fix the parser, potentially purge parser cache, and then you are 100% certain your wiki is clean. My second reason for being skeptical is im mostly unclear on what the benefits are over wikitext storage (this is the first time ive heard of the ve w/o parsoid thing. Are there other benefits? Simplifying parser cache by not having parser cache?) I may be misinterpreting how such a thing is proposed to work. Im not very familar with parsoid and associated things. --bawolff

Brian Wolff

10 p.m.

New subject: MediaWiki 2.0 (was: No more Architecture Committee?)

On Jan 20, 2015 5:53 PM, "Brian Wolff" <bawolff(a)gmail.com> wrote:

...

On Jan 20, 2015 4:22 PM, "James Forrester" <jforrester(a)wikimedia.org>

wrote:

...

> > On 20 January 2015 at 12:04, Jeroen De Dauw <jeroendedauw(a)gmail.com>

wrote:

...

> > > > > > - Get rid of wikitext on the server-side. > > > - HTML storage only. Remove MWParser from the codebase. All > > > extensions that hook into wikitext (so, almost all of them?)

will

...

> > > need to > > > be re-written. > > > > > > > Just to confirm: this is not actually on the WMF roadmap right? :) > > > > It's certainly not what I'm working on for the next year or so. It is > unlikely to be something we do for WMF usage; it's more valuable to

people

...

that want VisualEditor but want PHP-only (or don't want Node) for the server. J. --

Hypothetically how would this work? Wouldnt you still need parsoid to

verify the html corresponds to some legit wikitext? Otherwise how would you know its safe?

...

Since we are somewhat having a discussion about this (i recognize that

this isnt a "real" discussion in the sense that there isnt a full technical proposal, or any concrete plans to actually do it in near future, just a wild idea that some people like), one of the reasons im somewhat skeptical about this idea, is if there is an xss issue, you seem much more screwed with html storage, since now the bad stuff is in the canonical representation, instead of wikitext storage where you can just fix the parser, potentially purge parser cache, and then you are 100% certain your wiki is clean.

...

My second reason for being skeptical is im mostly unclear on what the

benefits are over wikitext storage (this is the first time ive heard of the ve w/o parsoid thing. Are there other benefits? Simplifying parser cache by not having parser cache?)

...

I may be misinterpreting how such a thing is proposed to work. Im not

very familar with parsoid and associated things.

...

--bawolff

And the other thread which i hadn't read at the time of writing this answers my question in that html verification is something yet to be determined :) --bawolff

Helder .

10:05 p.m.

New subject: MediaWiki 2.0 (was: No more Architecture Committee?)

On Tue, Jan 20, 2015 at 7:53 PM, Brian Wolff <bawolff(a)gmail.com> wrote:

...

My second reason for being skeptical is im mostly unclear on what the benefits are over wikitext storage (this is the first time ive heard of the ve w/o parsoid thing. Are there other benefits? Simplifying parser cache by not having parser cache?)

I don't know if this helps but HTML-only wikis are on the roadmap at least since May 2014: https://www.mediawiki.org/w/index.php?title=Parsoid/Roadmap/2014_15#HTML-on… Helder

James HK

8:21 p.m.

New subject: MediaWiki 2.0 (was: No more Architecture Committee?)

Hi,

...

- Get rid of wikitext on the server-side. - HTML storage only. Remove MWParser from the codebase. All extensions that hook into wikitext (so, almost all of them?) will need to be re-written.

You gotta to be kidding. It seems we are a bit out of sync here with the real world for proposing (or demanding not sure if above is being seriously considered) something like "All extensions that hook into wikitex ... will need to be re-written.". Just because it seems favourable to the WMF cluster to run Parsoid services it doesn't mean anyone else needs a bunch of extra node.js servers to handle Parsoid just to store and manage some text in mediawiki. Cheers

Daniel Friesen

8:57 p.m.

New subject: MediaWiki 2.0 (was: No more Architecture Committee?)

On 2015-01-20 12:21 PM, James HK wrote:

...

Hi,

- Get rid of wikitext on the server-side. - HTML storage only. Remove MWParser from the codebase. All extensions that hook into wikitext (so, almost all of them?) will need to be re-written.

Parsoid, etc... can already handle parser functions and tag hooks. So I'm not certain but, he may be talking specifically about extensions that hook into WikiText to implement their own custom syntax like [[Foo::Bar]].

...

Just because it seems favourable to the WMF cluster to run Parsoid services it doesn't mean anyone else needs a bunch of extra node.js servers to handle Parsoid just to store and manage some text in mediawiki.

As explained in the other topic, VisualEditor can edit the DOM without Parsoid, so this doesn't mean everyone would have to have Parsoid servers. ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]

James HK

9:11 p.m.

New subject: MediaWiki 2.0 (was: No more Architecture Committee?)

Hi, On 1/21/15, Daniel Friesen <daniel(a)nadir-seen-fire.com> wrote:

...

On 2015-01-20 12:21 PM, James HK wrote:

Hi,

- Get rid of wikitext on the server-side. - HTML storage only. Remove MWParser from the codebase. All extensions that hook into wikitext (so, almost all of them?) will need to be re-written.

We certainty want to avoid a situation where we have to write spaghetti code just because of if ( $parser === 'Parsoid' ) { return $this->doSomethingToBeAbleToHandle( '[[Foo::Bar]]' ); } Cheers

3389

days inactive

3393

days old

wikitech-l@lists.wikimedia.org

Manage subscription

16 comments

10 participants

tags (0)

participants (10)

Brian Wolff
Daniel Friesen
David Gerard
Dmitriy Sintsov
Federico Leva (Nemo)
Helder .
James Forrester
James HK
Jay Ashworth
Jeroen De Dauw