Let's not let this discussion sidetrack into "shared hosting vs VMs (vs docker?)" --- there's another phabricator ticket and summit topic for that ( https://phabricator.wikimedia.org/T87774 and https://phabricator.wikimedia.org/T113210. I'd prefer to have discussion in *this* particular task/thread concentrate on: * Hey, we can have JavaScript and PHP in the same packaging system. What cool things might that enable? * Hey, we can have JavaScript and PHP running together in the same server. Perhaps some persistence-related issues with PHP can be made easier? * Hey, we can actually write *extensions for mediawiki-core* in JavaScript (or CoffeeScript, or...) now. Or run PHP code inside Parsoid. How could we use that? (Could it grow developer communities?) * How are parser extensions (like, say, WikiHiero, but there are lots of them) going to be managed in the long term? There are three separate codebases to hook right now. An extension like <gallery> might eventually need to hook the image thumbnail service, too. Do we have a plan? And the pro/anti-npm and pro/anti-docker and pro/anti-VM discussion can go into one of those other tasks. Thanks. --scott

I would very, *very* much prefer to not have MediaWiki core extensions written in JavaScript. Even beyond my criticisms of JavaScript as a language, I feel like that just unnecessarily introduces complexity. The purpose of this wrapper is to combine separate micro-services that would otherwise be run in separate VMs / servers / etc. so that it can easily be run in a hosting setup. Otherwise, I'm interested in what implications this will have, especially for making MediaWiki easier to install and use, which would be awesome. --  Tyler Romeo https://parent5446.nyc 0x405D34A7C86B42DF From: C. Scott Ananian &lt;cananian(a)wikimedia.org&gt; Reply: Wikimedia developers &lt;wikitech-l(a)lists.wikimedia.org&gt; Date: November 6, 2015 at 14:14:13 To: Wikimedia developers &lt;wikitech-l(a)lists.wikimedia.org&gt; Subject:  Re: [Wikitech-l] [RFC/Summit] `npm install mediawiki-express` Let's not let this discussion sidetrack into "shared hosting vs VMs (vs docker?)" --- there's another phabricator ticket and summit topic for that ( https://phabricator.wikimedia.org/T87774 and https://phabricator.wikimedia.org/T113210. I'd prefer to have discussion in *this* particular task/thread concentrate on: * Hey, we can have JavaScript and PHP in the same packaging system. What cool things might that enable? * Hey, we can have JavaScript and PHP running together in the same server. Perhaps some persistence-related issues with PHP can be made easier? * Hey, we can actually write *extensions for mediawiki-core* in JavaScript (or CoffeeScript, or...) now. Or run PHP code inside Parsoid. How could we use that? (Could it grow developer communities?) * How are parser extensions (like, say, WikiHiero, but there are lots of them) going to be managed in the long term? There are three separate codebases to hook right now. An extension like <gallery> might eventually need to hook the image thumbnail service, too. Do we have a plan? And the pro/anti-npm and pro/anti-docker and pro/anti-VM discussion can go into one of those other tasks. Thanks. --scott _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

That's a pretty good point. Despite my comments, I'll definitely keep an open mind, and am interested in what people might propose. --  Tyler Romeo https://parent5446.nyc 0x405D34A7C86B42DF From: C. Scott Ananian &lt;cananian(a)wikimedia.org&gt; Reply: C. Scott Ananian &lt;cananian(a)wikimedia.org&gt; Date: November 6, 2015 at 15:01:59 To: Tyler Romeo &lt;tylerromeo(a)gmail.com&gt; CC: Wikimedia developers &lt;wikitech-l(a)lists.wikimedia.org&gt; Subject:  Re: [Wikitech-l] [RFC/Summit] `npm install mediawiki-express` Tyler: I hear you.  I'm not sure it's a good idea, either -- especially not for core extensions used in production. But it does perhaps allow some expansion of our developer community on the fringes, and makes writing extensions possible for a larger set of people?  And perhaps there are some cool things written in JavaScript which the extended community could more easily hook up to MediaWiki using `php-embed`. I'm not sure that there are.  I'm just opening up the discussion to see if anyone pipes up with, "oh, yeah, I've always wanted to do XYZ!". Greg: I agree re: premature stifling of discussion.  I'm just saying that "high-level" conversation is already happening elsewhere, and it's more productive there.  I started *this* particular thread trying to elicit discussion more narrowly focused on the thing I've just built.   --scott On Fri, Nov 6, 2015 at 2:30 PM, Tyler Romeo &lt;tylerromeo(a)gmail.com&gt; wrote: I would very, *very* much prefer to not have MediaWiki core extensions written in JavaScript. Even beyond my criticisms of JavaScript as a language, I feel like that just unnecessarily introduces complexity. The purpose of this wrapper is to combine separate micro-services that would otherwise be run in separate VMs / servers / etc. so that it can easily be run in a hosting setup. Otherwise, I'm interested in what implications this will have, especially for making MediaWiki easier to install and use, which would be awesome. --  Tyler Romeo https://parent5446.nyc 0x405D34A7C86B42DF From: C. Scott Ananian &lt;cananian(a)wikimedia.org&gt; Reply: Wikimedia developers &lt;wikitech-l(a)lists.wikimedia.org&gt; Date: November 6, 2015 at 14:14:13 To: Wikimedia developers &lt;wikitech-l(a)lists.wikimedia.org&gt; Subject:  Re: [Wikitech-l] [RFC/Summit] `npm install mediawiki-express` Let's not let this discussion sidetrack into "shared hosting vs VMs (vs docker?)" --- there's another phabricator ticket and summit topic for that ( https://phabricator.wikimedia.org/T87774 and https://phabricator.wikimedia.org/T113210. I'd prefer to have discussion in *this* particular task/thread concentrate on: * Hey, we can have JavaScript and PHP in the same packaging system. What cool things might that enable? * Hey, we can have JavaScript and PHP running together in the same server. Perhaps some persistence-related issues with PHP can be made easier? * Hey, we can actually write *extensions for mediawiki-core* in JavaScript (or CoffeeScript, or...) now. Or run PHP code inside Parsoid. How could we use that? (Could it grow developer communities?) * How are parser extensions (like, say, WikiHiero, but there are lots of them) going to be managed in the long term? There are three separate codebases to hook right now. An extension like <gallery> might eventually need to hook the image thumbnail service, too. Do we have a plan? And the pro/anti-npm and pro/anti-docker and pro/anti-VM discussion can go into one of those other tasks. Thanks. --scott _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l -- (http://cscott.net)

On 2015-11-06 1:12 PM, Brion Vibber wrote: node has a built in vm <https://nodejs.org/api/vm.html> module that is regularly used to execute sandboxed js that doesn't have access to the privileged node api. This code doesn't have access to `process.exit()` and PHP's concept of fatal errors (in addition to thrown exceptions) that immediately halt the process and can't be caught doesn't exist in JS. Sandboxing against infinite loops could also be done by running the sandbox in another process (child_process even has a high-level message passing stream for communicating with a node js child process). That all being said. I still think the original rationale for picking lua (more sandboxing controls including execution limits based on steps in lua rather than varying execution time) is still valid. ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]

On Fri, Nov 6, 2015 at 4:12 PM, Brion Vibber &lt;bvibber(a)wikimedia.org&gt; wrote: Right, exactly. I'm currently running Opcache and APCu inside the embedded PHP which are going to some lengths to offer persistent caches. I'm not an expert on PHP architecture; I suspect there are other places in mediawiki where we are similarly jumping through hoops. Perhaps these could be simplified, at least for certain users. I expect that I can port php-embed to PHP 7 and/or HHVM without too much trouble, if interest warrants. And I already support quite a large number of different node versions, from 2.4.0 to 5.0.0. And there are some interesting other alternative implementations that could export the same interface but use RPC to bridge node and PHP, see for instance https://github.com/bergie/dnode-php. Even the sync/async distinction can be bridged; if you look at the underlying implementation for php-embed all communication is done via async message passing between the threads. We just "stop and wait" for certain replies to emulate sync calls (in particular for PHP, which prefers it that way). There is sandboxing within v8, so your warning is not accurate. And in fact, the "mirror image" project is the PHP extension v8js, which I believe Tim started and I worked on for a while before attempting node-php-embed. It also uses the native v8 sandboxing facilities. Yup. That's what we essentially already do: after starting to implement the template engine in Parsoid, it was scrapped and the entire templating engine is implemented by calling over to PHP to allow it to expand templates. And whenever we want more information about the expansion, we implement it in PHP. But that's essentially the genesis of the "mediawiki as a collection of services" idea -- once you start doing this, you find all sorts of bits of crufty complex PHP code which you'd rather not try to reimplement. First templates, then image thumbnailing, next who knows, probably the skin. One day they might all be spun out as separate services with internal HTTP calls between them. I'm just providing a PoC that lets you ask questions about potential alternatives. I welcome the discussion. Yeah. Ideally you should only have to write one implementation, and it should be It doesn't basically. It just asks PHP to do the expansion for it, and then wraps the whole thing in an element warning VE not to touch it. Except for citations. On our roadmap for this quarter we have a task to write a "proper" extension interface, and then use it to refactor the citation code and (hopefully) implement <gallery> support. The end goal being to empower the community to write Parsoid extensions for all the large number of *other* tag extensions we don't yet support. Note that Visual Editor needs to be extended at the same time as Parsoid is extended, so they continue to recognize the same DOM structures, and to make the resulting structures "easily editable" in some fashion.. The exact contours of all this is still a research project. This work is, again, a PoC to see if we could allow folks to package their Parsoid extension as an NPM module, and perhaps include in the same module the necessary VE code and hook it up. --scott -- (http://cscott.net)