[Moving threads for on-topic-ness.]
On 16 January 2015 at 07:01, Brian Wolff bawolff@gmail.com wrote:
Does anyone actually have anything they want that is difficult to do currently and requires a mass compat break?
Sure.
Three quick examples of things on the horizon (I'm not particularly saying we'd actually do these for Wikimedia's use, but if you're going to ask for straw man arguments… :-)):
- Get rid of wikitext on the server-side. - HTML storage only. Remove MWParser from the codebase. All extensions that hook into wikitext (so, almost all of them?) will need to be re-written. - Real-time collaborative editing. - Huge semantic change to the concept of a 'revision'; we'd probably need to re-structure the table from scratch. Breaking change for many tools in core and elsewhere. - Replace local PHP hooks with proper services interfaces instead. - Loads of opportunities for improvements here (anti-spam tools 'as a service', Wordpress style; pre-flighting saves; ), but again, pretty much everything will need re-writing; this would likely be "progressive", happening one at a time to areas where it's useful/wanted/needed, but it's still a huge breaking change for many extensions.
Proposing to rewrite mediawiki because we can without even a notion of what we would want to do differently seems silly.
Oh, absolutely. I think RobLa's point was that it's unclear who feels empowered to make that decision (rather than the pitch). I don't. I don't think RobLa does. Clearly the Architecture Committee don't.
J.
Why wikitext is so much disliked? It's more compact to type than HTML. It's a templating language. HTML is not. Then something like Handlebars (which is weaker than wikitext) should be used. Or, something like web components and custom tags. But why removing nice thing (wikitext) which saves a lot of keystrokes to break the whole codebase? Strange. Especially because there's more progressive Parsoid. It looks like whole MW can be rewritten in Node.js, not Python.
On Fri, Jan 16, 2015 at 8:04 PM, James Forrester jforrester@wikimedia.org wrote:
[Moving threads for on-topic-ness.]
On 16 January 2015 at 07:01, Brian Wolff bawolff@gmail.com wrote:
Does anyone actually have anything they want that is difficult to do currently and requires a mass compat break?
Sure.
Three quick examples of things on the horizon (I'm not particularly saying we'd actually do these for Wikimedia's use, but if you're going to ask for straw man arguments… :-)):
- Get rid of wikitext on the server-side.
extensions that hook into wikitext (so, almost all of them?) will
- HTML storage only. Remove MWParser from the codebase. All
need to be re-written.
- Real-time collaborative editing.
need to re-structure the table from scratch. Breaking change for
- Huge semantic change to the concept of a 'revision'; we'd probably
many tools in core and elsewhere.
- Replace local PHP hooks with proper services interfaces instead.
- Loads of opportunities for improvements here (anti-spam tools 'as a service', Wordpress style; pre-flighting saves; ), but again, pretty
much everything will need re-writing; this would likely be "progressive", happening one at a time to areas where it's useful/wanted/needed, but it's still a huge breaking change for many extensions.
Proposing to rewrite mediawiki because we can without even a notion of what we would want to do differently seems silly.
Oh, absolutely. I think RobLa's point was that it's unclear who feels empowered to make that decision (rather than the pitch). I don't. I don't think RobLa does. Clearly the Architecture Committee don't.
J.
James D. Forrester Product Manager, Editing Wikimedia Foundation, Inc.
jforrester@wikimedia.org | @jdforrester _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Jan 16, 2015 1:05 PM, "James Forrester" jforrester@wikimedia.org wrote:
[Moving threads for on-topic-ness.]
On 16 January 2015 at 07:01, Brian Wolff bawolff@gmail.com wrote:
Does anyone actually have anything they want that is difficult to do currently and requires a mass compat break?
Sure.
Three quick examples of things on the horizon (I'm not particularly
saying
we'd actually do these for Wikimedia's use, but if you're going to ask for straw man arguments… :-)):
- Get rid of wikitext on the server-side.
extensions that hook into wikitext (so, almost all of them?) will
- HTML storage only. Remove MWParser from the codebase. All
need to
be re-written.
- Real-time collaborative editing.
need to re-structure the table from scratch. Breaking change for
- Huge semantic change to the concept of a 'revision'; we'd probably
many tools in core and elsewhere.
- Replace local PHP hooks with proper services interfaces instead.
- Loads of opportunities for improvements here (anti-spam tools 'as a service', Wordpress style; pre-flighting saves; ), but again,
pretty much
everything will need re-writing; this would likely be "progressive", happening one at a time to areas where it'suseful/wanted/needed, but it's still a huge breaking change for many extensions.
Woo strawmen for me to shoot down! :)
Actually, The revision thing is fair. Its pretty engrained that a pages have linear revisions, and each one has a single author, allowing multiple authors or branching and remerging would probably be a big enough change to warrant calling it 2.0. And it kind of fits, since last time structure of revisions was really rearranged afaik was 1.5.
Without going into the merits/drawbacks of html storage - i dont see that being rewrite worthy. Whether the blob of data in the text table/es is html or wikitext doesnt really matter to mw. Especially with content handler.
Hooks are a very active area of code. The sort of area where i would guess that adding an extra 20ms of time per each hook invocation to call an external service would not be ok (since there are hundreds of hook calls in a request). I dont think the notion of a service fits with how hooks are used, at least for many hooks. Of course someone could make a heavier weight version of hooks for where its a good idea. Or even just a wrapper object that forwards things to a service. I dont think this is worthy of a 2.0.
--bawolff
----- Original Message -----
From: "James Forrester" jforrester@wikimedia.org
Three quick examples of things on the horizon (I'm not particularly saying we'd actually do these for Wikimedia's use, but if you're going to ask for straw man arguments… :-)):
- Get rid of wikitext on the server-side.
Are we still litigating "should we piss off the power user editors* in the service of making things easier for notional new ones"?
Or have I misunderstood the suggestion?
Cheers, -- jra [* understanding that that's not the *only* thing which killing off wikitext breaks, merely the one that came first to mind ]
On 2015-01-18 5:29 PM, Jay Ashworth wrote:
----- Original Message -----
From: "James Forrester" jforrester@wikimedia.org Three quick examples of things on the horizon (I'm not particularly saying we'd actually do these for Wikimedia's use, but if you're going to ask for straw man arguments… :-)):
- Get rid of wikitext on the server-side.
Are we still litigating "should we piss off the power user editors* in the service of making things easier for notional new ones"?
Or have I misunderstood the suggestion?
Cheers, -- jra [* understanding that that's not the *only* thing which killing off wikitext breaks, merely the one that came first to mind ]
Parsoid can do Parsoid DOM to WikiText conversions. So I believe the suggestion is that storage be switched entirely to the Parsoid DOM and WikiText in classic editing just becomes a method of editing the content that is stored as Parsoid DOM in the backend.
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]
----- Original Message -----
From: "Daniel Friesen" daniel@nadir-seen-fire.com
On 2015-01-18 5:29 PM, Jay Ashworth wrote:
----- Original Message -----
From: "James Forrester" jforrester@wikimedia.org Three quick examples of things on the horizon (I'm not particularly saying we'd actually do these for Wikimedia's use, but if you're going to ask for straw man arguments… :-)):
- Get rid of wikitext on the server-side.
Are we still litigating "should we piss off the power user editors* in the service of making things easier for notional new ones"?
Or have I misunderstood the suggestion?
Parsoid can do Parsoid DOM to WikiText conversions. So I believe the suggestion is that storage be switched entirely to the Parsoid DOM and WikiText in classic editing just becomes a method of editing the content that is stored as Parsoid DOM in the backend.
Aha! Sorry; I was behind on Parsoid, was the problem. Yeah, if there's a way to edit in MWtext, both for humans and programs, then that serves the use cases I would be concerned about. Thanks for the prompt clarification, Daniel.
Cheers, -- jra
On 19 January 2015 at 01:46, Jay Ashworth jra@baylink.com wrote:
Aha! Sorry; I was behind on Parsoid, was the problem. Yeah, if there's a way to edit in MWtext, both for humans and programs, then that serves the use cases I would be concerned about. Thanks for the prompt clarification, Daniel.
There's also fourteen years of legacy page versions which I think absolutely nobody is going to convert.
- d.
On 19 January 2015 at 05:43, David Gerard dgerard@gmail.com wrote:
On 19 January 2015 at 01:46, Jay Ashworth jra@baylink.com wrote:
Aha! Sorry; I was behind on Parsoid, was the problem. Yeah, if there's a way to edit in MWtext, both for humans and programs, then that serves
the
use cases I would be concerned about. Thanks for the prompt
clarification,
Daniel.
There's also fourteen years of legacy page versions which I think absolutely nobody is going to convert.
Code is very good at "just doing it" if we wanted to do it. A few billion revisions are just ephemeral details if it actually makes sense to change over. Not that we should do it blindly, but this shouldn't hold us back.
J.
Really, James? WMF still has to cleanup the conversion from UseModWiki: https://phabricator.wikimedia.org/T2323 When bug 323 and others are fixed, I may believe you about billions-revisions conversions.
Nemo
Hey,
FYI: we had this discussion for SMW some time ago. We where at version 1.9, following a naming pattern very similar to the MediaWiki one. The question there was if the next release containing a break should be either 1.10 (no change), 2.0 (following semver) or 10.0 (dropping the "1."). People generally preferred 2.0, mostly because "10.0 seems like such a big jump". So this is what we went with. And while the 2.0 release was not more significant than 1.9 or 1.8, no one seems to have gotten seriously confused about it.
- Get rid of wikitext on the server-side.
- HTML storage only. Remove MWParser from the codebase. All extensions that hook into wikitext (so, almost all of them?) willneed to be re-written.
Just to confirm: this is not actually on the WMF roadmap right? :)
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate Evil software architect at Wikimedia Germany ~=[,,_,,]:3
On 20 January 2015 at 12:04, Jeroen De Dauw jeroendedauw@gmail.com wrote:
>
- Get rid of wikitext on the server-side.
- HTML storage only. Remove MWParser from the codebase. All extensions that hook into wikitext (so, almost all of them?) willneed to be re-written.
Just to confirm: this is not actually on the WMF roadmap right? :)
It's certainly not what I'm working on for the next year or so. It is unlikely to be something we do for WMF usage; it's more valuable to people that want VisualEditor but want PHP-only (or don't want Node) for the server.
J.
On Jan 20, 2015 4:22 PM, "James Forrester" jforrester@wikimedia.org wrote:
On 20 January 2015 at 12:04, Jeroen De Dauw jeroendedauw@gmail.com
wrote:
>
- Get rid of wikitext on the server-side.
- HTML storage only. Remove MWParser from the codebase. All extensions that hook into wikitext (so, almost all of them?)
will
need to be re-written.
Just to confirm: this is not actually on the WMF roadmap right? :)
It's certainly not what I'm working on for the next year or so. It is unlikely to be something we do for WMF usage; it's more valuable to people that want VisualEditor but want PHP-only (or don't want Node) for the server.
J.
Hypothetically how would this work? Wouldnt you still need parsoid to verify the html corresponds to some legit wikitext? Otherwise how would you know its safe?
Since we are somewhat having a discussion about this (i recognize that this isnt a "real" discussion in the sense that there isnt a full technical proposal, or any concrete plans to actually do it in near future, just a wild idea that some people like), one of the reasons im somewhat skeptical about this idea, is if there is an xss issue, you seem much more screwed with html storage, since now the bad stuff is in the canonical representation, instead of wikitext storage where you can just fix the parser, potentially purge parser cache, and then you are 100% certain your wiki is clean.
My second reason for being skeptical is im mostly unclear on what the benefits are over wikitext storage (this is the first time ive heard of the ve w/o parsoid thing. Are there other benefits? Simplifying parser cache by not having parser cache?)
I may be misinterpreting how such a thing is proposed to work. Im not very familar with parsoid and associated things.
--bawolff
On Jan 20, 2015 5:53 PM, "Brian Wolff" bawolff@gmail.com wrote:
On Jan 20, 2015 4:22 PM, "James Forrester" jforrester@wikimedia.org
wrote:
On 20 January 2015 at 12:04, Jeroen De Dauw jeroendedauw@gmail.com
wrote:
>
- Get rid of wikitext on the server-side.
- HTML storage only. Remove MWParser from the codebase. All extensions that hook into wikitext (so, almost all of them?)
will
need to be re-written.
Just to confirm: this is not actually on the WMF roadmap right? :)
It's certainly not what I'm working on for the next year or so. It is unlikely to be something we do for WMF usage; it's more valuable to
people
that want VisualEditor but want PHP-only (or don't want Node) for the server.
J.
Hypothetically how would this work? Wouldnt you still need parsoid to
verify the html corresponds to some legit wikitext? Otherwise how would you know its safe?
Since we are somewhat having a discussion about this (i recognize that
this isnt a "real" discussion in the sense that there isnt a full technical proposal, or any concrete plans to actually do it in near future, just a wild idea that some people like), one of the reasons im somewhat skeptical about this idea, is if there is an xss issue, you seem much more screwed with html storage, since now the bad stuff is in the canonical representation, instead of wikitext storage where you can just fix the parser, potentially purge parser cache, and then you are 100% certain your wiki is clean.
My second reason for being skeptical is im mostly unclear on what the
benefits are over wikitext storage (this is the first time ive heard of the ve w/o parsoid thing. Are there other benefits? Simplifying parser cache by not having parser cache?)
I may be misinterpreting how such a thing is proposed to work. Im not
very familar with parsoid and associated things.
--bawolff
And the other thread which i hadn't read at the time of writing this answers my question in that html verification is something yet to be determined :)
--bawolff
On Tue, Jan 20, 2015 at 7:53 PM, Brian Wolff bawolff@gmail.com wrote:
My second reason for being skeptical is im mostly unclear on what the benefits are over wikitext storage (this is the first time ive heard of the ve w/o parsoid thing. Are there other benefits? Simplifying parser cache by not having parser cache?)
I don't know if this helps but HTML-only wikis are on the roadmap at least since May 2014: https://www.mediawiki.org/w/index.php?title=Parsoid/Roadmap/2014_15#HTML-onl...
Helder
Hi,
- Get rid of wikitext on the server-side.
extensions that hook into wikitext (so, almost all of them?) will need
- HTML storage only. Remove MWParser from the codebase. All
to be re-written.
You gotta to be kidding. It seems we are a bit out of sync here with the real world for proposing (or demanding not sure if above is being seriously considered) something like "All extensions that hook into wikitex ... will need to be re-written.". Just because it seems favourable to the WMF cluster to run Parsoid services it doesn't mean anyone else needs a bunch of extra node.js servers to handle Parsoid just to store and manage some text in mediawiki.
Cheers
On 2015-01-20 12:21 PM, James HK wrote:
Hi,
- Get rid of wikitext on the server-side.
extensions that hook into wikitext (so, almost all of them?) will need
- HTML storage only. Remove MWParser from the codebase. All
to be re-written.
You gotta to be kidding. It seems we are a bit out of sync here with the real world for proposing (or demanding not sure if above is being seriously considered) something like "All extensions that hook into wikitex ... will need to be re-written.".
Parsoid, etc... can already handle parser functions and tag hooks. So I'm not certain but, he may be talking specifically about extensions that hook into WikiText to implement their own custom syntax like [[Foo::Bar]].
Just because it seems favourable to the WMF cluster to run Parsoid services it doesn't mean anyone else needs a bunch of extra node.js servers to handle Parsoid just to store and manage some text in mediawiki.
As explained in the other topic, VisualEditor can edit the DOM without Parsoid, so this doesn't mean everyone would have to have Parsoid servers.
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]
Hi,
On 1/21/15, Daniel Friesen daniel@nadir-seen-fire.com wrote:
On 2015-01-20 12:21 PM, James HK wrote:
Hi,
- Get rid of wikitext on the server-side.
extensions that hook into wikitext (so, almost all of them?) will
- HTML storage only. Remove MWParser from the codebase. All
need to be re-written.
You gotta to be kidding. It seems we are a bit out of sync here with the real world for proposing (or demanding not sure if above is being seriously considered) something like "All extensions that hook into wikitex ... will need to be re-written.".
Parsoid, etc... can already handle parser functions and tag hooks. So I'm not certain but, he may be talking specifically about extensions that hook into WikiText to implement their own custom syntax like [[Foo::Bar]].
We certainty want to avoid a situation where we have to write spaghetti code just because of
if ( $parser === 'Parsoid' ) { return $this->doSomethingToBeAbleToHandle( '[[Foo::Bar]]' ); }
Cheers
wikitech-l@lists.wikimedia.org