Hi all,
There are several new & exciting developments in HTML templating land.
First of all, Ryan Kaldari kindly tested the Knockoff / TAssembly HTML templating library from a Mobile perspective [1,2], and summarized his findings on the wiki [3]. The main missing feature he identified was the ability to mark up control flow outside of HTML elements. To address this, I just added support for Knockout's comment syntax to the Knockoff compiler [4]. This turned out to be straightforward, and required no changes in the TAssembly runtime.
Secondly, Dan spotted two very promising developments in upstream KnockoutJS:
== Knockout Components == There's a new version of KnockoutJS coming that might bridge a gap we saw at the architecture summit in January, the one between templating and widget...ing. Knockout components [5,6] could be said bridge, they're basically web components but possible now.
== Curly braces == The other thing they're adding is text interpolation so you can do <span>{{name}}</span> and people don't have to be sad about Knockout's HTML attribute syntax. This can simplify the migration for existing handlebars compatible templates as used by Mobile & Flow.
The KnockoutJS release is currently in early beta [7], with the release expected later this summer.
Gabriel & Dan
[1] https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library/... [2] https://github.com/gwicke/knockoff/, https://github.com/gwicke/tassembly/ [3] https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library/... [4] https://github.com/gwicke/knockoff/commit/0e516979c3d1fb35489c58e7b659c3ad0b... [5] http://www.knockmeout.net/2014/06/knockout-3-2-preview-components.html [6] http://blog.stevensanderson.com/2014/06/11/architecting-large-single-page-ap... [7] https://github.com/knockout/knockout/tree/gh-pages-3.2.0
I just want to be clear that any sort of template syntax that resembles or can be confused with wikitext is not something that we can or should allow in core. If MobileFrontend and whatnot want to use this syntax, so be it.
-- Tyler Romeo 0x405D34A7C86B42DF
From: Gabriel Wicke gwicke@wikimedia.org Reply: Wikimedia developers wikitech-l@lists.wikimedia.org> Date: July 7, 2014 at 21:28:33 To: Wikimedia developers wikitech-l@lists.wikimedia.org> Subject: [Wikitech-l] HTML templating progress; Knockout Components & curly brace syntax
== Curly braces == The other thing they're adding is text interpolation so you can do <span>{{name}}</span> and people don't have to be sad about Knockout's HTML attribute syntax. This can simplify the migration for existing handlebars compatible templates as used by Mobile & Flow.
On Tue, Jul 8, 2014 at 7:16 AM, Tyler Romeo tylerromeo@gmail.com wrote:
I just want to be clear that any sort of template syntax that resembles or can be confused with wikitext is not something that we can or should allow in core. If MobileFrontend and whatnot want to use this syntax, so be it.
We can absolutely allow it, so that's not strictly true. Whether we should is another matter. I just want to be clear that this is your opinion.
-Chad
On Tue, Jul 8, 2014 at 10:23 AM, Chad innocentkiller@gmail.com wrote:
On Tue, Jul 8, 2014 at 7:16 AM, Tyler Romeo tylerromeo@gmail.com wrote:
I just want to be clear that any sort of template syntax that resembles
or
can be confused with wikitext is not something that we can or should
allow
in core. If MobileFrontend and whatnot want to use this syntax, so be it.
We can absolutely allow it, so that's not strictly true. Whether we should is another matter. I just want to be clear that this is your opinion.
I'm not sure I follow, it seems to me knockout syntax and wikitext are different, and have different purposes. Can you elaborate a bit more, Tyler?
In general, the syntax is different, which is why Knockout is a great solution. But using curly braces for variable insertion is similar to wikisyntax’s template transclusion. That alone is not really enough to cause a problem, but if we proceed in that general direction it could lead to issues. -- Tyler Romeo 0x405D34A7C86B42DF
From: Dan Andreescu dandreescu@wikimedia.org Reply: Wikimedia developers wikitech-l@lists.wikimedia.org> Date: July 8, 2014 at 10:27:08 To: Wikimedia developers wikitech-l@lists.wikimedia.org> Subject: Re: [Wikitech-l] HTML templating progress; Knockout Components & curly brace syntax
I'm not sure I follow, it seems to me knockout syntax and wikitext are different, and have different purposes. Can you elaborate a bit more, Tyler?
On Tue, 08 Jul 2014 16:16:06 +0200, Tyler Romeo tylerromeo@gmail.com wrote:
I just want to be clear that any sort of template syntax that resembles or can be confused with wikitext is not something that we can or should allow in core. If MobileFrontend and whatnot want to use this syntax, so be it.
That is probably too sternly worded, but I kind of agree that we should try to avoid ambiguities, especially if the templating is to ever be available to gadgets/user scripts/wiki page "templates".
We're already using wikitext as a very limited template engine in some places in core (most noticeable in localisation messages, but I think also also directly in some places), I think making the new one clearly distinct would be worthwhile.
This is exciting stuff. In similar news Flow and mobile now have a shared ResourceLoaderTemplateModule [1] that supports shipping templates from server to client.
As a next step we should explore making sure these two things are compatible.
It would also be highly useful when it is in a stable form to run some benchmarks against hogan and mustache on rendering time for some complex templates.
[1] http://MediaWiki.org/wiki/Extension:Mantle On 7 Jul 2014 18:28, "Gabriel Wicke" gwicke@wikimedia.org wrote:
Hi all,
There are several new & exciting developments in HTML templating land.
First of all, Ryan Kaldari kindly tested the Knockoff / TAssembly HTML templating library from a Mobile perspective [1,2], and summarized his findings on the wiki [3]. The main missing feature he identified was the ability to mark up control flow outside of HTML elements. To address this, I just added support for Knockout's comment syntax to the Knockoff compiler [4]. This turned out to be straightforward, and required no changes in the TAssembly runtime.
Secondly, Dan spotted two very promising developments in upstream KnockoutJS:
== Knockout Components == There's a new version of KnockoutJS coming that might bridge a gap we saw at the architecture summit in January, the one between templating and widget...ing. Knockout components [5,6] could be said bridge, they're basically web components but possible now.
== Curly braces == The other thing they're adding is text interpolation so you can do <span>{{name}}</span> and people don't have to be sad about Knockout's HTML attribute syntax. This can simplify the migration for existing handlebars compatible templates as used by Mobile & Flow.
The KnockoutJS release is currently in early beta [7], with the release expected later this summer.
Gabriel & Dan
[1]
https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library/... [2] https://github.com/gwicke/knockoff/, https://github.com/gwicke/tassembly/ [3]
https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library/... [4]
https://github.com/gwicke/knockoff/commit/0e516979c3d1fb35489c58e7b659c3ad0b... [5] http://www.knockmeout.net/2014/06/knockout-3-2-preview-components.html [6]
http://blog.stevensanderson.com/2014/06/11/architecting-large-single-page-ap... [7] https://github.com/knockout/knockout/tree/gh-pages-3.2.0
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 07/08/2014 07:28 AM, Jon Robson wrote:
This is exciting stuff. In similar news Flow and mobile now have a shared ResourceLoaderTemplateModule [1] that supports shipping templates from server to client.
As a next step we should explore making sure these two things are compatible.
Yes, that would be useful. IIRC Mantle already supports JSON templates, so this sounds fairly doable. Is there anything else that's missing?
It would also be highly useful when it is in a stable form to run some benchmarks against hogan and mustache on rendering time for some complex templates.
We have a suite of micro benchmarks at https://github.com/gwicke/TemplatePerf. Could you nominate or directly add some benchmark tasks that are interesting to you and are not yet covered by the existing ones? We should also add hogan. Handlebars and mustache are already covered.
Thanks!
Gabriel
Tyler: What issues might you foresee these leading to...? PHP and JavaScript have similar syntax and we use both of those...
Gabriel: It doesn't yet support JSON templates, but there is no reason it can't. We'd just need to send a different content type header.
In terms of benchmarks, I was thinking more along the lines of benchmarking on the client, in particular for mobile devices which may have less memory. This would essentially be, time to load the template from the server to the time taken to render.
On Tue, Jul 8, 2014 at 8:08 AM, Gabriel Wicke gwicke@wikimedia.org wrote:
On 07/08/2014 07:28 AM, Jon Robson wrote:
This is exciting stuff. In similar news Flow and mobile now have a shared ResourceLoaderTemplateModule [1] that supports shipping templates from server to client.
As a next step we should explore making sure these two things are compatible.
Yes, that would be useful. IIRC Mantle already supports JSON templates, so this sounds fairly doable. Is there anything else that's missing?
It would also be highly useful when it is in a stable form to run some benchmarks against hogan and mustache on rendering time for some complex templates.
We have a suite of micro benchmarks at https://github.com/gwicke/TemplatePerf. Could you nominate or directly add some benchmark tasks that are interesting to you and are not yet covered by the existing ones? We should also add hogan. Handlebars and mustache are already covered.
Thanks!
Gabriel
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 07/08/2014 10:27 AM, Jon Robson wrote:
Tyler: What issues might you foresee these leading to...? PHP and JavaScript have similar syntax and we use both of those...
Gabriel: It doesn't yet support JSON templates, but there is no reason it can't. We'd just need to send a different content type header.
In terms of benchmarks, I was thinking more along the lines of benchmarking on the client, in particular for mobile devices which may have less memory. This would essentially be, time to load the template from the server to the time taken to render.
Makes sense.
Gabriel
On Tue, Jul 8, 2014 at 10:27 AM, Jon Robson jdlrobson@gmail.com wrote:
In terms of benchmarks, I was thinking more along the lines of benchmarking on the client, in particular for mobile devices which may have less memory. This would essentially be, time to load the template from the server to the time taken to render.
Is there someone who has the time to perform and publish an independent audit on performance specifically? We currently have this: https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#...
...which suggests that Knockoff is 50% faster than then next fastest solution (Handlebars), and over 10x faster than most. If this stands up to scrutiny, and holds true as Knockoff gains feature parity, this is a huge achievement that we should shout about far and wide, since I believe no one disputes that a DOM-based approach is much more secure than a string-based approach. Before bragging about this, though, we should make absolutely sure that it stands up to independent scrutiny.
Rob
On Tue, Jul 8, 2014 at 12:20 PM, Rob Lanphier robla@wikimedia.org wrote:
Is there someone who has the time to perform and publish an independent audit on performance specifically? We currently have this:
https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#...
...which suggests that Knockoff is 50% faster than then next fastest solution (Handlebars), and over 10x faster than most. If this stands up to scrutiny, and holds true as Knockoff gains feature parity, this is a huge achievement that we should shout about far and wide, since I believe no one disputes that a DOM-based approach is much more secure than a string-based approach...
Yes, it's 50% faster, but we're not really comparing apples to apples. Knockout/T-assembly requires a pre-compilation step, so we should keep that in mind in the comparisons. The pre-compilation doesn't affect performance, but it does add a layer of complexity to the process.
The approach that Mantle is taking is pretty interesting, i.e. allowing you to use any templating system you want as long as it meets certain assumptions. I would love to see Knockout/T-assembly and Mantle move towards being compatible with each other (which wouldn't take much work). Then people who want maximum performance/security can use Knockout and people who want maximum simplicity can use Mustache, etc. I know there are probably few people on this list who would think that offering simplicity is important, but we should remember that we are an open source project with thousands of 3rd party users, many of whom are not professional programmers, but would love to be able to tweak the interface and build simple extensions. Adopting an MVC framework with templating/widgets is a huge step in the right direction, but we would be squandering the opportunity, IMO, if our only templating system is too complicated for novice programmers to figure out how to use. If we can offer the best of both worlds, that seems like the ideal solution to me.
Ryan Kaldari
For all that is holy, we should pick one templating engine (that must be compatible with both JS and PHP) and switch everything to it. Having many templating engines would be horrible.
On Tue, Jul 8, 2014 at 12:56 PM, Ryan Kaldari rkaldari@wikimedia.org wrote:
On Tue, Jul 8, 2014 at 12:20 PM, Rob Lanphier robla@wikimedia.org wrote:
Is there someone who has the time to perform and publish an independent audit on performance specifically? We currently have this:
https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#...
...which suggests that Knockoff is 50% faster than then next fastest solution (Handlebars), and over 10x faster than most. If this stands up
to
scrutiny, and holds true as Knockoff gains feature parity, this is a huge achievement that we should shout about far and wide, since I believe no
one
disputes that a DOM-based approach is much more secure than a
string-based
approach...
Yes, it's 50% faster, but we're not really comparing apples to apples. Knockout/T-assembly requires a pre-compilation step, so we should keep that in mind in the comparisons. The pre-compilation doesn't affect performance, but it does add a layer of complexity to the process.
The approach that Mantle is taking is pretty interesting, i.e. allowing you to use any templating system you want as long as it meets certain assumptions. I would love to see Knockout/T-assembly and Mantle move towards being compatible with each other (which wouldn't take much work). Then people who want maximum performance/security can use Knockout and people who want maximum simplicity can use Mustache, etc. I know there are probably few people on this list who would think that offering simplicity is important, but we should remember that we are an open source project with thousands of 3rd party users, many of whom are not professional programmers, but would love to be able to tweak the interface and build simple extensions. Adopting an MVC framework with templating/widgets is a huge step in the right direction, but we would be squandering the opportunity, IMO, if our only templating system is too complicated for novice programmers to figure out how to use. If we can offer the best of both worlds, that seems like the ideal solution to me.
Ryan Kaldari _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Tue, Jul 8, 2014 at 12:56 PM, Ryan Kaldari rkaldari@wikimedia.org wrote:
On Tue, Jul 8, 2014 at 12:20 PM, Rob Lanphier robla@wikimedia.org wrote:
Is there someone who has the time to perform and publish an independent audit on performance specifically? We currently have this:
https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#...
...which suggests that Knockoff is 50% faster than then next fastest solution (Handlebars), and over 10x faster than most. If this stands up
to
scrutiny, and holds true as Knockoff gains feature parity, this is a huge achievement that we should shout about far and wide, since I believe no
one
disputes that a DOM-based approach is much more secure than a
string-based
approach...
Yes, it's 50% faster, but we're not really comparing apples to apples. Knockout/T-assembly requires a pre-compilation step, so we should keep that in mind in the comparisons. The pre-compilation doesn't affect performance, but it does add a layer of complexity to the process.
At a minimum, it would seem to affect deployment performance. We are much more forgiving of things that slow down deployment than we are of things that slow down typical page loads, but not infinitely forgiving. How much overhead would this add to deployments?
Template compilation is hardly a new technique (Smarty anyone?), so I'm assuming that technique could be (and has been?) applied to other engines.
The approach that Mantle is taking is pretty interesting, i.e. allowing you
to use any templating system you want as long as it meets certain assumptions. I would love to see Knockout/T-assembly and Mantle move towards being compatible with each other (which wouldn't take much work). Then people who want maximum performance/security can use Knockout and people who want maximum simplicity can use Mustache, etc. I know there are probably few people on this list who would think that offering simplicity is important, but we should remember that we are an open source project with thousands of 3rd party users, many of whom are not professional programmers, but would love to be able to tweak the interface and build simple extensions. Adopting an MVC framework with templating/widgets is a huge step in the right direction, but we would be squandering the opportunity, IMO, if our only templating system is too complicated for novice programmers to figure out how to use. If we can offer the best of both worlds, that seems like the ideal solution to me.
I think I'm with Max on this one. There can be only one. Or two. Definitely not more than two. Ok, maybe for some applications a third. But only some applications. Wait, where was I? Oh yeah, there can be only one. :-)
I'm personally somewhat skeptical about Knockoff, but only somewhat. My main concern is that we don't end up adopting something that people will consider to be yet another weird Mediawiki-ism, because we have a lot of those. If we're going to get into the templating engine business, we should play to win. That means having something that gets a full-throated endorsement from *at least* this community rather than "it's so great, it's one of three approved templating approaches in our codebase". We should also have something that has all of the ingredients to resonate with a larger development community than just ours.
Gabriel makes a compelling argument that a DOM-based approach is superior to string concatenation. A general argument against DOM-based approaches is performance, but Knockoff seems to have mitigated performance issues so well that it actually seems to perform better than most template engines. So, I'm left generally asking the question "what's the catch?"
It sounds like the catch may be complexity, but we shouldn't handwave about that. Ryan, what's an example of something that's complicated in Knockoff that's much simpler in Handlebars?
Rob
On 07/08/2014 03:04 PM, Rob Lanphier wrote:
At a minimum, it would seem to affect deployment performance. We are much more forgiving of things that slow down deployment than we are of things that slow down typical page loads, but not infinitely forgiving. How much overhead would this add to deployments?
This depends on how we wire this up. There would be no effect if compilation & caching happens dynamically in ResourceLoader, or a couple of ms per template if it's done on deploy.
Template compilation is hardly a new technique (Smarty anyone?), so I'm assuming that technique could be (and has been?) applied to other engines.
Indeed. Most fast engines are using compilation, and some like handlebars and knockoff also use pre-compilation for best performance.
I'm personally somewhat skeptical about Knockoff, but only somewhat. My main concern is that we don't end up adopting something that people will consider to be yet another weird Mediawiki-ism, because we have a lot of those.
I wholeheartedly agree. The idea behind Knockoff is to complement a popular and rich reactive library (KnockoutJS) with a fast single-shot runtime (TAssembly) for server-side and simple client-side use cases. Both KnockoutJS and Knockoff / TAssembly are general-purpose templating libraries without ties to MediaWiki.
Gabriel
Right now there are two things I would consider "complicated" about Knockoff/T-assembly: 1. Currently the developer has to set up and run the pre-compilation process themselves outside of MediaWiki (through a node.js interface). If we can get MediaWiki to handle this part (via ResourceLoader), it would hide that layer of complexity from the developer. This would be similar to what we did for LESS support. 2. Text nodes have to be constructed from within the surrounding tags. This isn't super complicated, but it's definitely unintuitive. Apparently, this may no longer be an issue soon, as Knockout is introducing curly-brace syntax, which is more widely known and intuitive for building text nodes.
Ryan Kaldari
On Tue, Jul 8, 2014 at 4:10 PM, Gabriel Wicke gwicke@wikimedia.org wrote:
On 07/08/2014 03:04 PM, Rob Lanphier wrote:
At a minimum, it would seem to affect deployment performance. We are
much
more forgiving of things that slow down deployment than we are of things that slow down typical page loads, but not infinitely forgiving. How
much
overhead would this add to deployments?
This depends on how we wire this up. There would be no effect if compilation & caching happens dynamically in ResourceLoader, or a couple of ms per template if it's done on deploy.
Template compilation is hardly a new technique (Smarty anyone?), so I'm assuming that technique could be (and has been?) applied to other
engines.
Indeed. Most fast engines are using compilation, and some like handlebars and knockoff also use pre-compilation for best performance.
I'm personally somewhat skeptical about Knockoff, but only somewhat. My main concern is that we don't end up adopting something that people will consider to be yet another weird Mediawiki-ism, because we have a lot of those.
I wholeheartedly agree. The idea behind Knockoff is to complement a popular and rich reactive library (KnockoutJS) with a fast single-shot runtime (TAssembly) for server-side and simple client-side use cases. Both KnockoutJS and Knockoff / TAssembly are general-purpose templating libraries without ties to MediaWiki.
Gabriel
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Both handlebars (JS) and lightncandy (handlebars reimplemented in PHP) support pre-compilation. Are the times in https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#... for pre-compilation?
Flow's handlebars templates are pre-compiled into PHP for server-side templating. We haven't yet tried pre-compiling them in JS. Ideally the API in Mantle[1] to render a template wouldn't change.
... if compilation & caching happens dynamically in ResourceLoader
Would be nice. The Handlebars compiler (into JS) and Knockoff/T-assembly compiler (into JSON) are both Node.js programs.
[1] https://www.mediawiki.org/wiki/Extension:Mantle#Developer_features -- =S Page Features engineer
On 07/09/2014 12:32 AM, S Page wrote:
Both handlebars (JS) and lightncandy (handlebars reimplemented in PHP) support pre-compilation. Are the times in https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#... for pre-compilation?
No, those timings currently include compiling the templates from scratch once per test run. We can add some runs with precompilation as well. While most of the speed-up for browser users is achieved by reducing the download size (no compiler to download), it'll still be interesting to see how much cpu time precompilation saves.
I just updated the results with the latest full result set from ruthenium:
https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#...
They now also includes Hogan.js and the Spacebars & HTMLJS benchmarks added by CScott.
... if compilation & caching happens dynamically in ResourceLoader
Would be nice.
Agreed.
Gabriel
On 07/08/2014 12:20 PM, Rob Lanphier wrote:
On Tue, Jul 8, 2014 at 10:27 AM, Jon Robson jdlrobson@gmail.com wrote:
In terms of benchmarks, I was thinking more along the lines of benchmarking on the client, in particular for mobile devices which may have less memory. This would essentially be, time to load the template from the server to the time taken to render.
Is there someone who has the time to perform and publish an independent audit on performance specifically? We currently have this: https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library#...
...which suggests that Knockoff is 50% faster than then next fastest solution (Handlebars), and over 10x faster than most. If this stands up to scrutiny, and holds true as Knockoff gains feature parity, this is a huge achievement that we should shout about far and wide, since I believe no one disputes that a DOM-based approach is much more secure than a string-based approach. Before bragging about this, though, we should make absolutely sure that it stands up to independent scrutiny.
The JS implementation is around 50% faster in some micro benchmarks, and about even with handlebars in others. The benchmark code is all on github [1], so please study, run & improve it.
I'm currently re-running the benchmarks using the latest code on ruthenium (checkout in /home/gwicke/TemplatePerf). Here are the first results:
Set: Knockoff (node.js) (knockoff-node) Test: test1 [050] Avg: 0.0815 Min: 0.0530 Max: 0.1050 Test: test1b [050] Avg: 0.1334 Min: 0.0760 Max: 0.1510 Test: test2 [050] Avg: 0.8882 Min: 0.6440 Max: 1.1260 Test: test2 lambda [050] Avg: 0.6492 Min: 0.4730 Max: 0.7880 Test: test3 [050] Avg: 0.3166 Min: 0.1960 Max: 0.5110 Set: Handlebars (node.js) (handlebars-node) Test: test1 [050] Avg: 0.2093 Min: 0.1270 Max: 0.2750 Test: test1b [050] Avg: 0.2911 Min: 0.1660 Max: 0.3850 Test: test2 [050] Avg: 0.8271 Min: 0.6050 Max: 1.0400 Test: test2 lambda [050] Avg: 1.4567 Min: 1.1430 Max: 2.0150 Test: test3 [050] Avg: 0.2920 Min: 0.1870 Max: 0.3680
So it looks like a slight slow-down in knockoff compared to the last run. Which makes sense, as these tests include the compilation to a JSON template in the benchmark, and the compiler has gained some features.
It would also be great if somebody could run the benchmarks with the very latest HHVM nightly. Ruthenium is still on precise, which misses some boost dependencies needed for the latest version. The version installed on ruthenium is from January.
Gabriel
wikitech-l@lists.wikimedia.org