Just a heads-up, folks:
Later this week I'm going to be installing a fix for some longstanding bugs with HTML in wiki pages. One of the issues is a difference in rendering of templates that improperly nest HTML tags, which caused certain badly-written templates to render in one way on Wikipedia but wildly broken on most other wikis.
(Don't forget that an important part of what Wikipedia & its sister projects are about is making information sharable and reusable. If the code breaks when copied to another site, that's Bad.)
Once the fix is in, templates should render about the same on Wikipedia and other wikis where the "HTML Tidy" plugin isn't being used. The bad news is that some of these templates will be broken; so it would be great if we can make sure they get cleaned up.
Problem templates are mostly those which start an HTML tag in one template, then finish it in another. For instance if {{table-header}} contains a <table>, and then the table rows and final </table> are in another template entirely. These have always broken on regular MediaWiki -- the template is required to close its own tags -- though they sometimes appeared to work on Wikipedia due to bugs with our HTML handling when Tidy is enabled.
I've done some automated checks on templates on en.wikipedia.org to make a list of likely problem candidates:
http://leuksman.com/misc/templates/html-table-start.txt http://leuksman.com/misc/templates/html-table-end.txt http://leuksman.com/misc/templates/html-table-row.txt
If you've got a template that you're not sure if it will work, try copying it to my test wiki at http://test.leuksman.com/ . This has the fix installed with the corrected behavior, so you can see about how it will render on Wikipedia next week.
Please make sure this information is disseminated to the various other language and project wikis people are working on; I don't want to hear "waah! all my templates broke and no one told me!" next week. :)
For the moment the same kind of construct with wiki tables ({| ... |}) will still work, but note that some time in the future we're going to have to look at 'fixing' that too. (This might require some enhancements to how templates work to make it easier to fill in long tables.)
-- brion vibber (brion @ pobox.com)
On 6/6/06, Brion Vibber brion@pobox.com wrote:
I've done some automated checks on templates on en.wikipedia.org to make a list of likely problem candidates:
http://leuksman.com/misc/templates/html-table-start.txt http://leuksman.com/misc/templates/html-table-end.txt http://leuksman.com/misc/templates/html-table-row.txt
Looks like the third list is largely composed of templates that use raw HTML table markup inside {{#if}} (which, coincidentally, includes a great many of the most widely-used infoboxes). This is going to be fun! ;-)
Kirill Lokshin wrote:
On 6/6/06, Brion Vibber brion@pobox.com wrote:
I've done some automated checks on templates on en.wikipedia.org to make a list of likely problem candidates:
http://leuksman.com/misc/templates/html-table-start.txt http://leuksman.com/misc/templates/html-table-end.txt http://leuksman.com/misc/templates/html-table-row.txt
Looks like the third list is largely composed of templates that use raw HTML table markup inside {{#if}} (which, coincidentally, includes a great many of the most widely-used infoboxes). This is going to be fun! ;-)
Infoboxes are a very important application of templates. There needs to be some way conditionally add table rows based on whether or not a given template parameter is defined. When people complained about the argument separators in {{#if:}} clashing with table syntax, I told them to use HTML table tags instead. That seemed like a much better, more stable solution than {{!}}. If that's not going to work anymore, what exactly are people meant to do?
There are thousands of affected articles, and if there is no solution for this, then I don't think it is acceptable to put this change live.
Why is the template required to close its own tags? It would make things a lot easier if this was not required.
-- Tim Starling
Tim Starling wrote:
Infoboxes are a very important application of templates. There needs to be some way conditionally add table rows based on whether or not a given template parameter is defined. When people complained about the argument separators in {{#if:}} clashing with table syntax, I told them to use HTML table tags instead. That seemed like a much better, more stable solution than {{!}}. If that's not going to work anymore, what exactly are people meant to do?
There are thousands of affected articles, and if there is no solution for this, then I don't think it is acceptable to put this change live.
Why is the template required to close its own tags? It would make things a lot easier if this was not required.
That's how MediaWiki has worked for a long time; it was just broken with Tidy mode enabled, thus guaranteeing that any Wikipedia page using such a template would be broken when copied to another wiki.
Would you prefer to change the behavior? Note that this will require some restructuring to the parser to both be more correct and and not break things, and will leave us with inconsistent, unparseable code in the future. (That is, it'll be impossible to tell what the code after a template inclusion will parse as unless the template is available.)
But if we're really, really sure, we can put some time into working on that and accept that our syntax will never be predictable. (This has consequences for future wysiwyg or markup-sensitive assisted editing plugins.)
-- brion vibber (brion @ pobox.com)
Brion Vibber wrote:
Tim Starling wrote:
There are thousands of affected articles, and if there is no solution for this, then I don't think it is acceptable to put this change live.
Why is the template required to close its own tags? It would make things a lot easier if this was not required.
That's how MediaWiki has worked for a long time; it was just broken with Tidy mode enabled, thus guaranteeing that any Wikipedia page using such a template would be broken when copied to another wiki.
Would you prefer to change the behavior? Note that this will require some restructuring to the parser to both be more correct and and not break things, and will leave us with inconsistent, unparseable code in the future. (That is, it'll be impossible to tell what the code after a template inclusion will parse as unless the template is available.)
But if we're really, really sure, we can put some time into working on that and accept that our syntax will never be predictable. (This has consequences for future wysiwyg or markup-sensitive assisted editing plugins.)
What I can do is for now separate the HTML whitelisting and HTML nesting correction into two separate parts, as it's (IMO incorrectly) done right now in Tidy mode. This will let all 1.7 wikis display these templates the same as Wikipedia currently does, by essentially copying the current (broken) Tidy behavior to the default install.
Then we can fight about this again in a few months. :)
-- brion vibber (brion @ pobox.com)
Brion Vibber wrote:
Would you prefer to change the behavior? Note that this will require some restructuring to the parser to both be more correct and and not break things, and will leave us with inconsistent, unparseable code in the future. (That is, it'll be impossible to tell what the code after a template inclusion will parse as unless the template is available.)
But if we're really, really sure, we can put some time into working on that and accept that our syntax will never be predictable. (This has consequences for future wysiwyg or markup-sensitive assisted editing plugins.)
Isn't it general practice with a macro-like facility like templates not to parse until you've substituted in the preprocessing phase? That is, {{subst:foo}}, {{foo}}, and copying and pasting in the code from Template:foo (and substituting the parameters manually) should all produce equivalent output. Sort of like C macros---it's madness to try to parse C code without first running the preprocessor.
If they're to have some sort of different semantics, that's a different paradigm of sorts---it would require no longer using templates as macros, but as functions that return complete well-formed sub-parts of the page. I'm not really opposed to that change (I think it could be made to work, and as you note, it makes some future editing tools easier to write), but it should then be made obvious and well-understood that templates aren't macros.
-Mark
Delirium wrote:
it should then be made obvious and well-understood that templates aren't macros.
I was going to add that we might want to do something about subst: then, too. IMO it's *very* counterintuitive that {{subst:foo}} and {{foo}} would render differently, especially since most of the documentation treats them as semantically equivalent.
-Mark
Delirium wrote:
Delirium wrote:
it should then be made obvious and well-understood that templates aren't macros.
I was going to add that we might want to do something about subst: then, too. IMO it's *very* counterintuitive that {{subst:foo}} and {{foo}} would render differently, especially since most of the documentation treats them as semantically equivalent.
Most of the time they will render the same.
The issue is when some smart-ass comes along and abuses the system; how badly will it break?
-- brion vibber (brion @ pobox.com)
Delirium wrote:
Brion Vibber wrote:
But if we're really, really sure, we can put some time into working on that and accept that our syntax will never be predictable. (This has consequences for future wysiwyg or markup-sensitive assisted editing plugins.)
Isn't it general practice with a macro-like facility like templates not to parse until you've substituted in the preprocessing phase?
That depends on the macro-like facility.
If they're to have some sort of different semantics, that's a different paradigm of sorts---it would require no longer using templates as macros, but as functions that return complete well-formed sub-parts of the page.
In what way is that not like a macro?
Note that the C preprocessor is not the only macro language in existence; it's just a particularly ugly and poorly-integrated example. In fact it's a perfect example of how *not* to design a macro language, since it's impossible to reliably work with C source code programatically due to it. Simple cases mostly work, while mildly complex cases explode dramatically for no apparent reason.
Dumb source-code replacement (like the C preprocessor) is extremely difficult to work with; however parse-tree replacement is clean and easy to work with for analysis, conversion, and editing tools.
-- brion vibber (brion @ pobox.com)
On 6/6/06, Brion Vibber brion@pobox.com wrote:
Problem templates are mostly those which start an HTML tag in one template, then finish it in another. For instance if {{table-header}} contains a <table>, and then the table rows and final </table> are in another template entirely. These have always broken on regular MediaWiki -- the template is required to close its own tags -- though they sometimes appeared to work on Wikipedia due to bugs with our HTML handling when Tidy is enabled.
Does this affect templates that use a <div> tag in one and a </div> tag in another? This affects many voting archive templates, such as {{at}}/{{ab}} or {{rfaf}}/{{rfab}}.
Death Phoenix wrote:
On 6/6/06, Brion Vibber brion@pobox.com wrote:
Problem templates are mostly those which start an HTML tag in one template, then finish it in another. For instance if {{table-header}} contains a <table>, and then the table rows and final </table> are in another template entirely. These have always broken on regular MediaWiki -- the template is required to close its own tags -- though they sometimes appeared to work on Wikipedia due to bugs with our HTML handling when Tidy is enabled.
Does this affect templates that use a <div> tag in one and a </div> tag in another? This affects many voting archive templates, such as {{at}}/{{ab}} or {{rfaf}}/{{rfab}}.
Yes.
-- brion vibber (brion @ pobox.com)
On 6/9/06, Brion Vibber brion@pobox.com wrote:
Death Phoenix wrote:
On 6/6/06, Brion Vibber brion@pobox.com wrote:
Problem templates are mostly those which start an HTML tag in one template, then finish it in another. For instance if {{table-header}} contains a
<table>, >> and >> then the table rows and final </table> are in another template entirely. >> These >> have always broken on regular MediaWiki -- the template is required to >> close its >> own tags -- though they sometimes appeared to work on Wikipedia due to >> bugs with >> our HTML handling when Tidy is enabled. >> > > Does this affect templates that use a <div> tag in one and a </div> tag in > another? This affects many voting archive templates, such as {{at}}/{{ab}} > or {{rfaf}}/{{rfab}}.
Yes.
-- brion vibber (brion @ pobox.com)
Hmmm... these templates are almost always substed rather than being transcluded. Will substing the template serve as a workaround for this?
On 6/9/06, Death Phoenix originaldeathphoenix@gmail.com wrote:
Hmmm... these templates are almost always substed rather than being transcluded. Will substing the template serve as a workaround for this?
Naturally. Substing any template is equivalent to typing in its text by hand. If typing <DIV> foo </DIV> is legal, then typing {{subst:divstart}} foo {{subst:divend}} is legal...
Steve
On 6/9/06, Steve Bennett stevagewp@gmail.com wrote:
On 6/9/06, Death Phoenix originaldeathphoenix@gmail.com wrote:
Hmmm... these templates are almost always substed rather than being transcluded. Will substing the template serve as a workaround for this?
Naturally. Substing any template is equivalent to typing in its text by hand. If typing <DIV> foo </DIV> is legal, then typing {{subst:divstart}} foo {{subst:divend}} is legal...
Wunderbar.