It sounds pretty much like the two templating RFCs are asking for the same exact thing, except just using different templating engines as the solution. If we want a comprehensive templating system in PHP core, Mustache is not going to be enough (I've used it before, and dealing with the lack of control structures is frustrating unless the templates are really simple). So I'd support using Twig or something similar to it like jinja2. Either way the RFCs can definitely be combined.

*-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2016 Major in Computer Science

On Thu, Dec 26, 2013 at 9:09 PM, Jon Robson jdlrobson@gmail.com wrote:

+1 to creating a new RFC that satisfies the needs of all 3. Although not mentioned Kaldari's proposal highlights a big existing need in MobileFrontend. Here we use JavaScript templates extensively and are now finding situations where we have the same HTML being generated on the server and client (for example after an edit the entire page is re rendered on the client).

I'm very excited to discuss this and push these RFCs along. Please feel free to bring me into any conversations. On 26 Dec 2013 18:04, "Rob Lanphier" robla@wikimedia.org wrote:

...

Hi folks,

This is an attempt to summarize the list of RFCs that are listed under this cluster:

https://www.mediawiki.org/wiki/Architecture_Summit_2014/RFC_Clusters#HTML_te...

...

...and possibly get a conversation going about all of this in advance of the Architecture Summit.

The main focus of all of these RFCs is around HTML generation for user interface elements. This category is not about wikitext templates or anything to do with how we translate wikitext markup

"Template Engine" is Chris Steipp's submission outlining the use of Twig. From my conversations with Chris, it's not so much that he's eager to adopt Twig specifically so much as standardize on *something*, and make sure there's usage guidelines around it to avoid common mistakes. He's seen many attempts at per-extension template libraries that bloat our code and often start off with big security vulnerabilities. There are many extensions that use Twig, and it seems to be a popular choice for new work, so Chris is hoping to standardize on it and put some usage standards around it.

"HTML templating library" is Ryan Kaldari's submission, promoting the use of Mustache or something like it. His main motivation is to have a Javascript template library for front-end work, but is hoping we choose something that has both Javascript and PHP implementations so that any PHP template system implementation is compatible with what we use for Javascript templating.

"MVC Framework" is Owen Davis's description of Wikia's Nirvana framework, which has been central to all of the user-facing work they've been doing for the past 2-3 years. As Owen points out in the talk page for this, it's really view-controller rather than full MVC. A big plus of adopting this RFC is that it would make it much more likely that Wikia-developed extensions (of which there are many) would be of greater use to the larger MediaWiki community, and would generally help facilitate greater collaboration between Wikia and the rest of us.

"OutputPage Refactor" is another submission from Owen Davis, which isn't really about templating, so much as taking the HTML generation code out of OutputPage. Wikia has been maintaining a fork of OutputPage for quite some time, so they already have quite a bit of experience with the proposed changes. This is clustered with the templating proposals, since I imagine the work that gets factored out of OutputPage would need to be factored into whatever templating system we choose.

The first three seem somewhat mutually exclusive, though it's clear our task is likely to come up with a fourth proposal that incorporates many requirements of those three. The OutputPage Refactor proposal, given some fleshing out, may not be controversial at all.

Where should we go from here? Can we make some substantial progress on moving one or more of these RFCs over the next few weeks?

Rob

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

I want a templating system that can be used both in PHP and JavaScript and fits in our way of doing i18n. And a bunny.

I'm not sure if this was meant as sarcastic but I do want this too and think it is a reasonable achievable goal - bunny optional!

On Fri, Dec 27, 2013 at 1:30 PM, Chad innocentkiller@gmail.com wrote:

On Fri, Dec 27, 2013 at 12:34 PM, Jon Robson jdlrobson@gmail.com wrote:

...

...

I want a templating system that can be used both in PHP and JavaScript

and

...

fits in our way of doing i18n. And a bunny.

I'm not sure if this was meant as sarcastic but I do want this too and think it is a reasonable achievable goal - bunny optional!

Bunnies should be listed in the requirements ;-)

I believe unicorns were in the requirements for search.

In all seriousness, PHP, JavaScript, and fitting i18n sound like minimum requirements. I'd also throw in html escaping by default.

Nik

On Fri, Dec 27, 2013 at 12:42 PM, Jon Robson jdlrobson@gmail.com wrote:

We updated our RFC:

https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#...

I started a list of requirements, nice-to-haves, and a table of the different templating libraries available and what their features are. Unfortunately, I don't know anything about Twig or Nirvana, so it would be great if other people could help flesh out this table before the Architecture summit.

Ryan Kaldari

Ha. Well, we’ve been using mustache at Wikia for a while, but not heavily. The backstory is that we’ve written approximately 270 extensions over the years and we have written a few simple frameworks to keep that code manageable. One of those things is a pseudo-mvc framework. The template itself is not an object, and it can be either a PHP or mustache template. A mustache template has more limitations, but in many cases it is sufficient and in general I feel like it’s a better idea to just pass data to a simple template rather than having a lot of logic in the template. Additionally, being able to share the same template between front end and back end has advantages.

In my opinion, backwards compatibility with the existing quick template stuff is impossible to avoid, so retaining a PHP option is essential but the mustache option gives a lot of power to the front end developers which is where a lot of the work is being done now…

For example, the modal dialog that pops up at the end of the new wiki creation process is a mustache template:

https://github.com/Wikia/app/blob/dev/extensions/wikia/CreateNewWiki/templat...

On Dec 27, 2013, at 8:41 AM, Niklas Laxström niklas.laxstrom@gmail.com wrote:

2013/12/27 Tyler Romeo tylerromeo@gmail.com

...

If we want a comprehensive templating system in PHP core,

I want a templating system that can be used both in PHP and JavaScript and fits in our way of doing i18n. And a bunny.

-Niklas _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Dec 27, 2013, at 8:27 PM, Matthew Walker mwalker@wikimedia.org wrote:

Owen,

That's really nifty actually -- can you talk a little about how y'all manage the i18n issues? Do you prerender all the messages outside of the template? Or do you use Mustache lambda's to do some formatting inside the template itself?

Hi, sorry for the delay. Holiday vacation. :)

We have a simple Controller framework to go along with the templates. When we originally proposed it we were going to put 100% of the logic and data inside the controller side of things and pass it to the template. However, that just creates more boilerplate/temp variables and makes the resulting code and templates a lot harder to read 6 months later. For example doing messages in the controller and passing the data to the template:

$message = wfMessage(‘message’); $template->setVal($message);

And then the template would have something like:

<?php echo $message ?>

vs: _nothing_ necessary in the controller and calling the message functions directly in the template:

<?php echo wfMessage(‘message’) ?>

In the first case you actually have no idea whether the source for $message is actually a mediawiki i18n message or something else, but in the second case it’s a lot more clear. And from a readability perspective the best code is no code at all. :)

So that’s why we put message calls in the template. Since we allow function calls, theoretically you can call any function, but we prevent the developers from doing that by shooting them with nerf darts when they try. Unlike the built in Quick template stuff, our templates are not objects, but they still have access to global variables and functions.

Additionally, where and how do you draw the line between using a template, and having additional logic outside of the template?

The only logic we put in templates is simple conditional stuff and loops. isset(), empty(), foreach() etc. No point in having a separate template language when you’re already doing PHP on the back end.

I'm also curious how you manage the coupling between layers -- do you pass in all variables to all templates; or is there some sort of management / documentation layer that specifies what data (and what format the data needs to be in) should be provided to a template.

The documentation is a bit of a mess, we just use PHP docstrings. We did originally propose a more strict validation/documentation framework to go along with it, and some of that code was written but we aren’t using it yet. The self-documenting nature of the Mediawiki API classes is a particularly nice feature, and we would have preferred to do something like that but we haven’t gotten around to it…

When writing very complex multi-function Special Pages (almost all of our internal tools are built as special pages) it gets kind of unwieldy with the special page class that just has a single execute() function and the redundant boilerplate to define ajax functions etc. Since most of our front end is javascript now and we sometimes want templates/html or json data from the same controllers, we have a 1:1 mapping between methods and templates and every controller method is automatically an ajax function that can return json or html. The front end can call ANY back end method and ask for json data or the html with the data applied to it. When the Controller gets “too unwieldy” (the threshold for this depends on the developer) we generally refactor them into a single Controller and a set of Helper classes so that we retain the same external end points, just moving the code around.

Here’s an example of that:

On every content page, there’s a right rail module that shows the latest photos uploaded to the wiki:

http://fallout.wikia.com/wiki/Nikola_Tesla_and_You_(Fallout_3)

on the back end, that’s entirely self contained in this LatestPhotosController which is dropped into the skin with a render() function. However, the data that it generates can be used in other places:

http://fallout.wikia.com/wikia.php?controller=LatestPhotos (call controller, combine data with html and return it) http://fallout.wikia.com/wikia.php?controller=LatestPhotos&format=json (call controller, just return the data that would have been in the template)

The default method is executeIndex() and the default template is <controller>_Index. Here’s the controller code:

https://github.com/Wikia/app/blob/dev/skins/oasis/modules/LatestPhotosContro...

And the template:

https://github.com/Wikia/app/blob/dev/skins/oasis/modules/templates/LatestPh...

Hope that helps provide a bit more context for how this is actually used in the application.

-Owen

~Matt Walker Wikimedia Foundation Fundraising Technology Team

On Fri, Dec 27, 2013 at 6:44 PM, Owen Davis owen@wikia-inc.com wrote:

...

Ha. Well, we’ve been using mustache at Wikia for a while, but not heavily. The backstory is that we’ve written approximately 270 extensions over the years and we have written a few simple frameworks to keep that code manageable. One of those things is a pseudo-mvc framework. The template itself is not an object, and it can be either a PHP or mustache template. A mustache template has more limitations, but in many cases it is sufficient and in general I feel like it’s a better idea to just pass data to a simple template rather than having a lot of logic in the template. Additionally, being able to share the same template between front end and back end has advantages.

In my opinion, backwards compatibility with the existing quick template stuff is impossible to avoid, so retaining a PHP option is essential but the mustache option gives a lot of power to the front end developers which is where a lot of the work is being done now…

For example, the modal dialog that pops up at the end of the new wiki creation process is a mustache template:

https://github.com/Wikia/app/blob/dev/extensions/wikia/CreateNewWiki/templat...

On Dec 27, 2013, at 8:41 AM, Niklas Laxström niklas.laxstrom@gmail.com wrote:

...

2013/12/27 Tyler Romeo tylerromeo@gmail.com

...

If we want a comprehensive templating system in PHP core,

I want a templating system that can be used both in PHP and JavaScript

and

...

fits in our way of doing i18n. And a bunny.

-Niklas _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

What sort of control structures would you want supported? Keep in mind that ideally we want something that is simple enough that it could be loaded as a client-side library on mobile, and also be reasonably sure that it won't introduce any security issues.

Ryan Kaldari

On Dec 27, 2013, at 7:53 AM, Tyler Romeo tylerromeo@gmail.com wrote:

It sounds pretty much like the two templating RFCs are asking for the same exact thing, except just using different templating engines as the solution. If we want a comprehensive templating system in PHP core, Mustache is not going to be enough (I've used it before, and dealing with the lack of control structures is frustrating unless the templates are really simple). So I'd support using Twig or something similar to it like jinja2. Either way the RFCs can definitely be combined.

*-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2016 Major in Computer Science

On Thu, Dec 26, 2013 at 9:09 PM, Jon Robson jdlrobson@gmail.com wrote:

...

+1 to creating a new RFC that satisfies the needs of all 3. Although not mentioned Kaldari's proposal highlights a big existing need in MobileFrontend. Here we use JavaScript templates extensively and are now finding situations where we have the same HTML being generated on the server and client (for example after an edit the entire page is re rendered on the client).

I'm very excited to discuss this and push these RFCs along. Please feel free to bring me into any conversations. On 26 Dec 2013 18:04, "Rob Lanphier" robla@wikimedia.org wrote:

...

Hi folks,

This is an attempt to summarize the list of RFCs that are listed under this cluster:

https://www.mediawiki.org/wiki/Architecture_Summit_2014/RFC_Clusters#HTML_te...

...

...and possibly get a conversation going about all of this in advance of the Architecture Summit.

The main focus of all of these RFCs is around HTML generation for user interface elements. This category is not about wikitext templates or anything to do with how we translate wikitext markup

"Template Engine" is Chris Steipp's submission outlining the use of Twig. From my conversations with Chris, it's not so much that he's eager to adopt Twig specifically so much as standardize on *something*, and make sure there's usage guidelines around it to avoid common mistakes. He's seen many attempts at per-extension template libraries that bloat our code and often start off with big security vulnerabilities. There are many extensions that use Twig, and it seems to be a popular choice for new work, so Chris is hoping to standardize on it and put some usage standards around it.

"HTML templating library" is Ryan Kaldari's submission, promoting the use of Mustache or something like it. His main motivation is to have a Javascript template library for front-end work, but is hoping we choose something that has both Javascript and PHP implementations so that any PHP template system implementation is compatible with what we use for Javascript templating.

"MVC Framework" is Owen Davis's description of Wikia's Nirvana framework, which has been central to all of the user-facing work they've been doing for the past 2-3 years. As Owen points out in the talk page for this, it's really view-controller rather than full MVC. A big plus of adopting this RFC is that it would make it much more likely that Wikia-developed extensions (of which there are many) would be of greater use to the larger MediaWiki community, and would generally help facilitate greater collaboration between Wikia and the rest of us.

"OutputPage Refactor" is another submission from Owen Davis, which isn't really about templating, so much as taking the HTML generation code out of OutputPage. Wikia has been maintaining a fork of OutputPage for quite some time, so they already have quite a bit of experience with the proposed changes. This is clustered with the templating proposals, since I imagine the work that gets factored out of OutputPage would need to be factored into whatever templating system we choose.

The first three seem somewhat mutually exclusive, though it's clear our task is likely to come up with a fourth proposal that incorporates many requirements of those three. The OutputPage Refactor proposal, given some fleshing out, may not be controversial at all.

Where should we go from here? Can we make some substantial progress on moving one or more of these RFCs over the next few weeks?

Rob

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

On Fri, Dec 27, 2013 at 11:58 AM, Ryan Kaldari rkaldari@wikimedia.orgwrote:

What sort of control structures would you want supported? Keep in mind that ideally we want something that is simple enough that it could be loaded as a client-side library on mobile, and also be reasonably sure that it won't introduce any security issues.

As in any type of control structure. Mustache is explicitly a template language without control structures. You can technically implement if/else statements and for loops using Mustache's sections, but you can't have expressions in your if statements. Template languages like Twig are more like an actual programming language.

Mustache also does not have template inheritance, which would be really useful in an extensible templating system.

On a side note, there are a lot of things that both Mustache and Twig do that are lot easier and more sensible in the latter. Filters, for example. In Mustache using filters involves using a section and looks exactly the same as an if statement. Using multiple filters at once involves multiple sections. In Twig there's a specific syntax that makes filtering a lot simpler. Also Twig has PhpStorm support, which is a plus.

*-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2016 Major in Computer Science

On Fri, Dec 27, 2013 at 4:54 PM, Ryan Kaldari rkaldari@wikimedia.orgwrote:

On Fri, Dec 27, 2013 at 4:36 PM, Tyler Romeo tylerromeo@gmail.com wrote:

...

As in any type of control structure. Mustache is explicitly a template language without control structures. You can technically implement

if/else

...

statements and for loops using Mustache's sections, but you can't have expressions in your if statements. Template languages like Twig are more like an actual programming language.

Having a template library that is essentially a programming language is something I would prefer to avoid. The entire point of using a template library (IMO) is to separate the markup from the logic. We can still use PHP or JS to process anything that needs serious munging.

I believe Twig does this better than Mustache. Twig does allow for complex for loops and if statements, but it's all about manipulating the rendered output. Try to think how you would implement a pager in Twig vs Mustache. In twig you can pass in a 'page' parameter and using callbacks manipulate the conditions of the foreach loop. In mustache an outside function would have to manipulate the data set before passing it in. Basically; I'm in the camp that the templating system should have access to all the data just in case it decides to use it rather than causing everyone to build frameworks around it.

Mustache also has another weakness that the above exposes -- In twig you must explicitly register each callback / filter allowing one to build a library. In Mustache you must pass anonymous functions to be passed in the data hash. So now the data to be rendered has code in it that's language and probably template dependent necessitating a preprocessing layer to inject those functions, or otherwise manipulate the data, depending on what language you're in.

Another couple of use cases we can analyse, that I feel are things that are properly in the display layer rather than having to manipulate them without being aware of display context in the backend (basically I'm making a case for decoupling):

For the fundraising thank you letters, I pass a dictionary to the template containing the currency string "VND 20000.23"; via callback that gets transformed into 20,000.23*₫* via a i18n library that I wrote.

I pass in details about a contribution (such as ID, date, email, name) to the template, and then these get composited into things like links, personalized greetings, and displayed tables.

~Matt

On 12/27/2013 08:48 PM, Matthew Walker wrote:

I believe Twig does this better than Mustache. Twig does allow for complex for loops and if statements, but it's all about manipulating the rendered output. Try to think how you would implement a pager in Twig vs Mustache. In twig you can pass in a 'page' parameter and using callbacks manipulate the conditions of the foreach loop. In mustache an outside function would have to manipulate the data set before passing it in.

This doesn't seem like a great use case. Almost any realistic pager is going to have too much data to send it all to the client. This even includes a pager of all the revisions of a single article (a lot of pagers in MW have far more pages than that).

That means the paging has to be done server side anyway.

Matt Flaschen

As in any type of control structure. Mustache is explicitly a template language without control structures. You can technically implement if/else statements and for loops using Mustache's sections, but you can't have expressions in your if statements. Template languages like Twig are more like an actual programming language.

This actually seems like a point against Twig for me. I really hope we don't introduce yet another programming language and added complexity on top of our already extremely complex stack....

That said I'd hope whatever solution we go with can be slotted out by 3rd parties in favour of another templating language.

My experience with mustache was that it was reasonably speedy, but that was with Python, java, and javascript libraries, not php.

I believe the default way of referencing text in mustache is to escape it. You have to add some special character to the reference to not escape it. I'm pretty sure I've done this one time in my whole life.

Sent from my iPhone

On Dec 27, 2013, at 12:09 PM, Chris Steipp csteipp@wikimedia.org wrote:

...

On Dec 27, 2013 7:54 AM, "Tyler Romeo" tylerromeo@gmail.com wrote:

It sounds pretty much like the two templating RFCs are asking for the same exact thing, except just using different templating engines as the solution. If we want a comprehensive templating system in PHP core, Mustache is not going to be enough (I've used it before, and dealing with the lack of control structures is frustrating unless the templates are really simple). So I'd support using Twig or something similar to it like jinja2. Either way the RFCs can definitely be combined.

Mustache is probably the second most popular engine that I hear people talk about. I would love to hear more details about what you see as it's shortcomings, or positive features that we would want to look for in whatever library we choose.

Kaldari's rfc about JavaScript is a use case I hadn't personally seen in any extensions that I reviewed. But anything to help with our pile of Dom xss vulnerabilities I would gladly support.

Are there any other libraries that have the same (or similar) syntax and security properties from php and JavaScript we should look at? I'm happy to research the security of any recommendations we get.

Obviously, along with security performance is the next big issue. It would be great if someone could work with Ori to get a feel of how any recommendations perform too. I did some very basic testing on twig. It would be great to see how mustache performes on similar tests.

...

*-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2016 Major in Computer Science

...

On Thu, Dec 26, 2013 at 9:09 PM, Jon Robson jdlrobson@gmail.com wrote:

+1 to creating a new RFC that satisfies the needs of all 3. Although not mentioned Kaldari's proposal highlights a big existing need in MobileFrontend. Here we use JavaScript templates extensively and are now finding situations where we have the same HTML being generated on the server and client (for example after an edit the entire page is re

rendered

...

...

on the client).

I'm very excited to discuss this and push these RFCs along. Please feel free to bring me into any conversations.

...

On 26 Dec 2013 18:04, "Rob Lanphier" robla@wikimedia.org wrote:

Hi folks,

This is an attempt to summarize the list of RFCs that are listed under this cluster:

https://www.mediawiki.org/wiki/Architecture_Summit_2014/RFC_Clusters#HTML_te...

...

...

...

...and possibly get a conversation going about all of this in advance of the Architecture Summit.

The main focus of all of these RFCs is around HTML generation for user interface elements. This category is not about wikitext templates or anything to do with how we translate wikitext markup

"Template Engine" is Chris Steipp's submission outlining the use of Twig. From my conversations with Chris, it's not so much that he's eager to adopt Twig specifically so much as standardize on *something*, and make sure there's usage guidelines around it to avoid common mistakes. He's seen many attempts at per-extension template libraries that bloat our code and often start off with big security vulnerabilities. There are many extensions that use Twig, and it seems to be a popular choice for new work, so Chris is hoping to standardize on it and put some usage standards around it.

"HTML templating library" is Ryan Kaldari's submission, promoting the use of Mustache or something like it. His main motivation is to have a Javascript template library for front-end work, but is hoping we choose something that has both Javascript and PHP implementations so that any PHP template system implementation is compatible with what we use for Javascript templating.

"MVC Framework" is Owen Davis's description of Wikia's Nirvana framework, which has been central to all of the user-facing work they've been doing for the past 2-3 years. As Owen points out in the talk page for this, it's really view-controller rather than full MVC. A big plus of adopting this RFC is that it would make it much more likely that Wikia-developed extensions (of which there are many) would be of greater use to the larger MediaWiki community, and would generally help facilitate greater collaboration between Wikia and the rest of us.

"OutputPage Refactor" is another submission from Owen Davis, which isn't really about templating, so much as taking the HTML generation code out of OutputPage. Wikia has been maintaining a fork of OutputPage for quite some time, so they already have quite a bit of experience with the proposed changes. This is clustered with the templating proposals, since I imagine the work that gets factored out of OutputPage would need to be factored into whatever templating system we choose.

The first three seem somewhat mutually exclusive, though it's clear our task is likely to come up with a fourth proposal that incorporates many requirements of those three. The OutputPage Refactor proposal, given some fleshing out, may not be controversial at all.

Where should we go from here? Can we make some substantial progress on moving one or more of these RFCs over the next few weeks?

Rob

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

---

Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l