tl;dr: We've been collectively whining about templates for long enough. Who wants to help with fixing them?
In the recent discussions/debacles about technical and stylistic advances, a recurring theme is that the use of some templates causes major headaches, and a commonly heard complaint from the developers and designers is that their products exhibit problems and shortcoming because of that. Anecdotal evidence I've lately encountered includes:
* The mobile skin obfuscates talk page access because the templates commonly found on talk pages makes them render horribly. * The mobile skin special-cases some templates (notably issue templates and infoboxes) because they would render horribly. * Media-viewer has a tough time doing to correct thing with attribution and license information because parsing template-madness is hard. * VE development has spent a large amount of time around templates, and it's still one of its weakest suits. Template substitution is still a problem, as well as templates that produce wikitext that in itself doesn't map cleanly to HTML tokens. * Scribunto has been developed specifically because writing and maintaining templates with more complicated logic is horrible, both from a writers/maintainers perspective as well as from a performance perspective
All this together is sufficient to assert we have a template problem. The main editing community has a problem with how templates are and must be used, the readers have a problem with display issues on mobile as well as style inconsistencies, the technical editing community has a problem with writing and maintaining templates, and the development community has a problem with the difficulty in correctly parsing and interpreting templates and there contents.
It would be great if this problem were tackled; it would be even greater if the WMF could work together with the community to identify the pain points, and jointly take steps to tackle them. Templates are currently extraordinarily powerful, and most if not all of this power is finding use in the projects, possibly in ways nobody ever foresaw. As we all know from Uncle Ben, with great power comes great responsibility, and it's about time we all took our share of that responsibility, tough up, and fix it.
We should keep in mind that current use is paramount, and any fixing of templates that breaks the wiki is frankly unacceptable, which probably means we can't go from insane to sane overnight, even if we could define sane and insane with regards to templates overnight. At the same time we shouldn't shy away from fixes that would break some exotic use of templates, if as part of the process of making things better, before implementation, we can fix those templates.
I hope we can, for the coming period, accomplish the following:
* Catalog the problems with templates. Make a comprehensive list that enumerates the problems with templates we have now, categories the problems (right now I'm roughly thinking in style, wikitext parsing rules and generated HTML, creation and writing issues (let's hope there is little of this one left after Scribunto), and usability by editors). * Note which quirks that lead to technical difficulties are used in the wild as features rather than bugs. * Brain storm possible (partial) solutions. * Find candidates that have high bang-for-buck possible solutions without impeding future improvements much. * Refine those solutions so we know quite exactly what it will fix, what it won't fix, and what it would possibly break. * Define sane fallback procedures for when things break; this should mainly come from the editing communities, but could probably use some guidance of what is possible/easy/logical/feasible from a technical POV from the development community. * Fix templates.
Personally, I'm all talk and no action, so to get this of the ground we would need a lot of help. First, we need to know if I'm on to something, or if this is just the raving of a lunatic (please tell me if it is!). If the idea is sound, we need to set up the infrastructure. We probably need a Meta page set up to organise things and set up initial reconnaissance. We need a lot of grunt work categorising issues and problems from all perspectives: reader (this is difficult, but many groups that don't directly represent the readers care deeply about their needs, so that's something), template users, template maintainers, and template infrastructure maintainers (developers). For that we need to reach out to those different communities; this email is posted to wikimedia-l only (because I couldn't think of a better one, but I acknowledge this doesn't fit like a glove), but there are bound to be other interested parties out there who want to help that this email isn't reaching.
What do you all think? Should we make this happen?
--Martijn
On 2 September 2014 10:40, Martijn Hoekstra martijnhoekstra@gmail.com wrote:
All this together is sufficient to assert we have a template problem.
...or to assert that we have a problem with new products being written that cope well with a theoretical ideal, but not with their real-world applications.
On Tue, Sep 2, 2014 at 5:40 AM, Martijn Hoekstra martijnhoekstra@gmail.com wrote:
tl;dr: We've been collectively whining about templates for long enough. Who wants to help with fixing them?
Improving on templates is broadly what I've been doing with my dialog tools https://en.wikinews.org/wiki/Help:Dialog, which I've been working on for about three years and am hoping to start using seriously pretty soon (I've got a few more tweaks in mind to do first; after that, further improvements I expect to be driven by experience with serious use). I've been developing these tools using javascript under the hood, although I'm sure they could in theory be done better as a wiki extension, because I reckoned the design would need to be able to turn on a dime and I'd observed the wiki-extension approval process was, well, not.
The dialog tools mediate passing data from page to page, using elements specified using wiki markup, with the particular intent that a wiki community could use these tools to crowdsource wizards https://en.wikipedia.org/wiki/Wizard_%28software%29 entirely written in wiki markup.
Data is entered using input elements such as text boxes and dropdown menus, then passed to the next page via buttons, all placed via templates like {{dialog/textarea}}, {{dialog/button}}. At the receiving page, dialog data can go into other input elements, but can also be substituted for template parameters, and expressions using template-expansion can be used to specify values for input elements, so that the whole thing meshes tolerably well with the template system rather than competing with it. Each {{dialog/button}} specifies an action to be performed. The usual action in the middle of a dialog is "view", to display a page, but there's also an action "edit" --- hedged around with safeguards against abuse, of course; safeguards predicated on the assumption that admins are trustworthy.
In designing dialog-action edit, I uncovered something curious that hints to me (as a programming-language designer) that the whole concept of wiki templates might have been subtly flawed... though I don't imagine the flaw could have been anticipated at the time, and I agree wholeheartedly that at this late date, not breaking things is paramount. With the API edit action, there's an optional preload page; and I'd originally imagined that for dialog edit I'd want to allow the preload page to take template parameters; but when I actually started creating dialog edit, keeping in mind the sorts of wiki activities I was familiar with that one might want a wizard for, I realized that it was more natural to provide a "form" page that would be fully template-expanded to produce the raw content for the edited page. In techspeak, this difference between preload pages with template-parameter substitution, versus dialog-edit forms with full template-expansion to generate raw content, is essentially that the preload page is a macro https://en.wikipedia.org/wiki/Macro_%28computer_science%29#Syntactic_macros, while the dialog edit form is a fexpr https://en.wikipedia.org/wiki/Fexpr.
On Tue, Sep 2, 2014 at 1:34 PM, pi zero wn.pi.zero@gmail.com wrote:
On Tue, Sep 2, 2014 at 5:40 AM, Martijn Hoekstra < martijnhoekstra@gmail.com> wrote:
tl;dr: We've been collectively whining about templates for long enough.
Who
wants to help with fixing them?
Improving on templates is broadly what I've been doing with my dialog tools https://en.wikinews.org/wiki/Help:Dialog, which I've been working on for about three years and am hoping to start using seriously pretty soon (I've got a few more tweaks in mind to do first; after that, further improvements I expect to be driven by experience with serious use). I've been developing these tools using javascript under the hood, although I'm sure they could in theory be done better as a wiki extension, because I reckoned the design would need to be able to turn on a dime and I'd observed the wiki-extension approval process was, well, not.
The dialog tools mediate passing data from page to page, using elements specified using wiki markup, with the particular intent that a wiki community could use these tools to crowdsource wizards https://en.wikipedia.org/wiki/Wizard_%28software%29 entirely written in wiki markup.
Data is entered using input elements such as text boxes and dropdown menus, then passed to the next page via buttons, all placed via templates like {{dialog/textarea}}, {{dialog/button}}. At the receiving page, dialog data can go into other input elements, but can also be substituted for template parameters, and expressions using template-expansion can be used to specify values for input elements, so that the whole thing meshes tolerably well with the template system rather than competing with it. Each {{dialog/button}} specifies an action to be performed. The usual action in the middle of a dialog is "view", to display a page, but there's also an action "edit" --- hedged around with safeguards against abuse, of course; safeguards predicated on the assumption that admins are trustworthy.
In designing dialog-action edit, I uncovered something curious that hints to me (as a programming-language designer) that the whole concept of wiki templates might have been subtly flawed... though I don't imagine the flaw could have been anticipated at the time, and I agree wholeheartedly that at this late date, not breaking things is paramount. With the API edit action, there's an optional preload page; and I'd originally imagined that for dialog edit I'd want to allow the preload page to take template parameters; but when I actually started creating dialog edit, keeping in mind the sorts of wiki activities I was familiar with that one might want a wizard for, I realized that it was more natural to provide a "form" page that would be fully template-expanded to produce the raw content for the edited page. In techspeak, this difference between preload pages with template-parameter substitution, versus dialog-edit forms with full template-expansion to generate raw content, is essentially that the preload page is a macro < https://en.wikipedia.org/wiki/Macro_%28computer_science%29#Syntactic_macros
,
while the dialog edit form is a fexpr <https://en.wikipedia.org/wiki/Fexpr
.
This is what I consider an excellent example of the power of templates, and the power we want to have editors to have.
At the same time, it's also a great demonstration why templates suck so much as a programming language. I can't really imagine anyone being able to quickly (or, honestly, slowly) make much sense of https://en.wikinews.org/w/index.php?title=Template:Dialog/button&action=... and the amount of hoops it requires you to jump through to do something useful, like make a button.
If I quickly scan it it seems to be a template that conditionally calls a subtemplate with different parameters, that calls a lua module which is an interpreter for some custom dialect of Lisp, which evaluates the string passed in. All tied together with javascript.
If we have to resort to such magic to make templates do what we want, templates are quite simply broken; how can we explain that to a newcomer. "To help with these templates, all you have to know about are wikitext templates, our own implementation of lisp, Javascript, and Lua, and you'll be good to go". I suspect the number of people in the world who know how to do that is very close to 1. Especially for usecases like this, we need something less complicated.
<This reply is still my own personal views, and in no way represents anything official>
On Tue, Sep 2, 2014 at 10:20 AM, Martijn Hoekstra <martijnhoekstra@gmail.com
wrote:
If we have to resort to such magic to make templates do what we want, templates are quite simply broken; how can we explain that to a newcomer. "To help with these templates, all you have to know about are wikitext templates, our own implementation of lisp, Javascript, and Lua, and you'll be good to go". I suspect the number of people in the world who know how to do that is very close to 1. Especially for usecases like this, we need something less complicated.
If "we" (TINW) actually want dialogs (which I'm not convinced of beyond a few very special cases), trying to do it in templates with embedded code is the wrong way to do it.
On Tue, Sep 2, 2014 at 4:46 PM, Brad Jorsch (Anomie) bjorsch@wikimedia.org wrote:
<This reply is still my own personal views, and in no way represents anything official>
On Tue, Sep 2, 2014 at 10:20 AM, Martijn Hoekstra < martijnhoekstra@gmail.com
wrote:
If we have to resort to such magic to make templates do what we want, templates are quite simply broken; how can we explain that to a newcomer. "To help with these templates, all you have to know about are wikitext templates, our own implementation of lisp, Javascript, and Lua, and
you'll
be good to go". I suspect the number of people in the world who know how
to
do that is very close to 1. Especially for usecases like this, we need something less complicated.
If "we" (TINW) actually want dialogs (which I'm not convinced of beyond a few very special cases), trying to do it in templates with embedded code is the wrong way to do it.
Well, that's the discussion I'm trying to open: If that's the wrong way, is there a right way (yet) ? I'd like to look at this from the perspective of what the correct way would be to make that happen.
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
On Tue, Sep 2, 2014 at 11:23 AM, Martijn Hoekstra <martijnhoekstra@gmail.com
wrote:
On Tue, Sep 2, 2014 at 4:46 PM, Brad Jorsch (Anomie) < bjorsch@wikimedia.org> wrote:
<This reply is still my own personal views, and in no way represents anything official>
On Tue, Sep 2, 2014 at 10:20 AM, Martijn Hoekstra < martijnhoekstra@gmail.com
wrote:
If we have to resort to such magic to make templates do what we want, templates are quite simply broken; how can we explain that to a
newcomer.
"To help with these templates, all you have to know about are wikitext templates, our own implementation of lisp, Javascript, and Lua, and
you'll
be good to go". I suspect the number of people in the world who know
how
to
do that is very close to 1. Especially for usecases like this, we need something less complicated.
If "we" (TINW) actually want dialogs (which I'm not convinced of beyond a few very special cases), trying to do it in templates with embedded code
is
the wrong way to do it.
Well, that's the discussion I'm trying to open: If that's the wrong way, is there a right way (yet) ? I'd like to look at this from the perspective of what the correct way would be to make that happen.
That would take a more detailed look at what is actually trying to be accomplished with these dialogs. For example, enwiki's Teahouse[1] has a dialog for newbies to more easily post questions, but it's implemented as a gadget that generates the form specific for that page. To what extent to we have actual use cases for many differently-designed dialogs? How does the lisp even enter into the current design?
But in general, the answer almost certainly isn't to be doing things with a mess of complex templates with their own custom lisp dialect.
[1]: https://en.wikipedia.org/wiki/Wikipedia:Teahouse/Questions
On Tue, Sep 2, 2014 at 12:00 PM, Brad Jorsch (Anomie) <bjorsch@wikimedia.org
wrote:
That would take a more detailed look at what is actually trying to be accomplished with these dialogs. For example, enwiki's Teahouse[1] has a dialog for newbies to more easily post questions, but it's implemented as a gadget that generates the form specific for that page. To what extent to we have actual use cases for many differently-designed dialogs? How does the lisp even enter into the current design?
The part about lisp, perhaps I've addressed in the other message I just posted (though it seems to have gone off with a distressing number of typos in it). And as for looking at examples of dialogs --- I agree, and I /don't/ have examples, exactly because this ****storm over superprotect has been precipitated by the Foundation /before/ I'd really intended to go widely public with my tools. I've decided people should be aware of what I'm doing even though I'm not quite there yet, but honestly, the primary route I'd always envisioned, to convince people that my ideas were viable, was to /demonstrate/ it, by building the tools and then starting to use them to build wizards. This is what has always worked best for me in the past: if I can see the potential in a low-level tool, the best way to show others the potential I see is to make it real. It's taken me an appalling amount of time (three years) to develop the tools, partly because at every little step in the design I've stopped to consider the implications for practical applications (like the difference between carving a statue by hand-and-eye with a hammer and chisel, versus pouring cement into a mold). Which I suppose is another part of my contention that design decisions about wikis should be made in the field.
To be clear, ordinary user of the dialog tools most certainly does *not* involve knowing javascript or Lua or html. The whole point is to arrange that an ordinary wiki users can do wonderful things using *only wiki markup*. (The hypothetical example of a url there is, btw, entirely wrong; but one wouldn't write any sort of url anyway.)
The syntax for specifying a typical button would be more like {{dialog/view|page::Name of page}}, but most people wouldn't even have to figure that much out for themselves --- they would copy-and-modify somebody else's markup they'd found somewhere that did something similar to what they wanted, and they'd use common sense for how to modify it. That's part of the great strength of wiki markup: you do simple things, and while doing them you see other, slightly more advanced things that others have done, so later on when you find you want some of those slightly more advanced things, you've seen them before and know where to look for examples.
Yes, there is in fact a lisp interpreter written in Lua. It makes up for some other weaknesses of templates. I expect most users not to have to touch that most of the time, and when they do they'll usually be able to just copy others' markup and make slight, obvious modifications. It's so vastly handy that I've used it (you've reminded me) in some of the dialog templates, just because it's so much easier to do things with it. I created it because it was obvious I'd need a powerful succinct way, without the cumbersome notational overhead of templates or Lua (or JavaScript, perish the thought), to specify transformations of the raw content of a wiki page; and the one place thus far where I use it intensively is for just such an application (removing from a wiki page all template calls of one kind, and modifying all those of another kind). I expect to be able to readily build tools, using dialogs and that lisp interpreter, to do things of the same general order as HotCat (only more sophisticated), and given the contrast between a few lines of Lisp and the JavaScript source code for HotCat, I think I'm quite justified in considering it an improvement... if it proves out. (There is some question whether this technique as currently implemented will really scale to manipulating the whole content of large wiki pages, due to practical limitations that may be imposed on the dialog tools by... the template-handling of the wiki software; but we'll see at what scale that becomes a problem, an what might be done under the hood to move the threshold when encountered.)
I do agree that templates are rather broken, although frankly the Foundation could --- with some deep insight --- have done things that would have improved them. The current plethora of magic words is a mess, partly because the template call-syntax is heavy and pure template/magic-word usage keeps sending things back to typeless text; both of which problems are not shared by my embedded lisp interpreter, with it's very-low-overhead call syntax and its simple, minimal, but highly useful dynamic type system.
<This reply still isn't anything official, but does represent my own views as a developer (both volunteer and staff)>
On Tue, Sep 2, 2014 at 12:30 PM, pi zero wn.pi.zero@gmail.com wrote:
Yes, there is in fact a lisp interpreter written in Lua. It makes up for some other weaknesses of templates. I expect most users not to have to touch that most of the time, and when they do they'll usually be able to just copy others' markup and make slight, obvious modifications. It's so vastly handy that I've used it (you've reminded me) in some of the dialog templates, just because it's so much easier to do things with it. I created it because it was obvious I'd need a powerful succinct way, without the cumbersome notational overhead of templates or Lua (or JavaScript, perish the thought), to specify transformations of the raw content of a wiki page;
Is there an actual problem with Scribunto that drove you to writing this lisp interpreter, or is it just that you don't like the fact that Scribunto forces you to separate Lua code from templates?
I consider this separation a good thing, as even if it is more work for the developer it makes things much easier for everyone else to understand later.
I do agree that templates are rather broken, although frankly the Foundation could --- with some deep insight --- have done things that would have improved them. The current plethora of magic words is a mess, partly because the template call-syntax is heavy and pure template/magic-word usage keeps sending things back to typeless text;
The Foundation *did* do something. It's called Scribunto.
On Tue, Sep 2, 2014 at 12:41 PM, Brad Jorsch (Anomie) <bjorsch@wikimedia.org
wrote:
<This reply still isn't anything official, but does represent my own views as a developer (both volunteer and staff)>
Is there an actual problem with Scribunto that drove you to writing this lisp interpreter, or is it just that you don't like the fact that Scribunto forces you to separate Lua code from templates?
I consider this separation a good thing, as even if it is more work for the developer it makes things much easier for everyone else to understand later.
I have a specific reason for disapproving of the separation. Wiki markup was incredibly successful; that's why there /is/ a wikimedian movement. It's not all been made to happen by flower-power. The two key characteristics of wiki markup that, I believe, made it work so spectacularly well are (1) It's very easy to use. It can get to be something of an awful mess at the high end, when the subtly design-flawed template mechanism gets forcibly overused, but when you get down to it, wiki markup is easy --- especially when one takes into account the second key characteristic, (2) it naturally promotes incremental learning. A while back I had a really simple edit to make to a Wikipedia page, and thought I'd try out VisualEditor on it. Eventually I gave up and did it by editing the markup. What I'd wanted to do was to add another simple item to an itemized list. The point here isn't how to do that, directly; the point is, even if I'd been a newbie with no clue how to do that, I'd have had no problem whatsoever with doing it in the absence of VE, because when I edited the markup, I'd have had examples right there in front of me of exactly how to do it. And as I think back, I learned just about everything I know about wiki markup that way. My first few edits were correcting typos, and I saw how wikilinks were done, and so on. At each stage, I'd seen examples of things just a little more advanced than what I'd done myself, because it was all right there in the markup I'd been editing, and when I wanted to do a little more than I had I'd go see how others had done similar things. But Lua is a completely different sort of language, requiring a major step --- and major steps don't belong to the wiki development pattern. Wikis work using volunteer labor because of their incremental approach, both to editing and to /learning/ to edit (and to learning how to participate in the community, but that's another subject).
And, Lua is a high-overhead language. You can't just write super-simple expressions and have them work, and build other stuff up. Lisp, on the other hand, has extremely simple, unambiguous, low-overhead syntax. (Even simple template calls have more syntactic overhead than lisp function calls.)
Of course, my primary goal was interactive pages, for building wizards, and it was clear to me that Lua wasn't going to help me with those anyway.
I do agree that templates are rather broken, although frankly the
Foundation could --- with some deep insight --- have done things that would have improved them. The current plethora of magic words is a mess, partly because the template call-syntax is heavy and pure template/magic-word usage keeps sending things back to typeless text;
The Foundation *did* do something. It's called Scribunto.
Scribunto which I maintain is, for the reasons I've attempted to articulate, philosophically incompatible with the factors that make wikis successful. I might add, that I believe these things are /still/ an essential ingredient for making wikis successful, and each time the wiki experience is changed in a way that moves away from those factors, the long-term success of the movement is degraded.
Scribunto isn't an improvement to templates; it's trying to half-replace them with something else. My understanding of "improve templates" would exclude a half-replacement (or full replacement), even if the half/full-replacement were to "improve *on* templates". (Actually, I do think Scribunto is an improvement on the status quo ante, but there are other routes that would have also been improvements on that, that I believe would have been better for the long-term health of the wikimedian movement.)
On Tue, Sep 2, 2014 at 1:35 PM, pi zero wn.pi.zero@gmail.com wrote:
(1) It's very easy to use.
[...]
(2) it naturally promotes incremental learning.
When it comes to your extremely complicated templates with embedded lisp, you're losing *both* of these. It's no longer very easy to use (although it seems you may consider cargo-cult programming to be "easy to use"), and your hypothetical "incremental learner" is more likely to be overwhelmed and give up rather than incrementally figure out where in some complicated mess filled with braces and parens the actual page content is located. We already have user studies (which I don't have the links for offhand) that show that people see the relatively simple infobox templates in articles and give up.
The advantage to having the complexity of Lua in modules is that the complexity specifically is hidden from new users. Once someone gets to the point where they're ready to make that step, they're probably motivated enough that looking through the manual (or even better would be if someone would write some good tutorials) isn't likely to be the barrier it would be to a complete newbie.
And, Lua is a high-overhead language. You can't just write super-simple expressions and have them work, and build other stuff up.
Sure you can; a little boilerplate is needed, but both that and the language itself are simple enough that someone could start from some basic examples and experiment easily enough. And we have a console on module edit pages that allows for direct experimentation with simple expressions.
Lisp, on the other hand, has extremely simple, unambiguous, low-overhead
syntax. (Even simple template calls have more syntactic overhead than lisp function calls.)
On the other hand, Lua's syntax and operation is similar to that of C, PHP, JavaScript, and so on, which are likely to be familiar to users with some sort of programming experience. While parser function syntax somewhat resembles Lisp, I'm not sure that similarity will help with understanding more than the surface.
I'm not also sure that Lisp's extremely simple syntax is necessarily an advantage. While extra syntactic structure requires extra learning, it also means that the written code is more structured which may be easier for humans to make sense of. Again, look at the trouble people report in understanding complicated templates.
Of course, my primary goal was interactive pages, for building wizards, and it was clear to me that Lua wasn't going to help me with those anyway.
I don't see how Lisp is helping that either. If you want interactive pages, JavaScript is the way to go. Or did you write your Lisp parser in JavaScript too?
Scribunto isn't an improvement to templates; it's trying to half-replace them with something else. My understanding of "improve templates" would exclude a half-replacement (or full replacement), even if the half/full-replacement were to "improve *on* templates".
Isn't your Lisp stuff also half-replacing templates with something else? Or are you overlooking that because Lisp syntax resembles that of templates?
Pardon my long reply; I actually meant to just comment on a couple of things, and got carried away. But I really do find this topic very interesting.
On Tue, Sep 2, 2014 at 2:03 PM, Brad Jorsch (Anomie) bjorsch@wikimedia.org wrote:
When it comes to your extremely complicated templates with embedded lisp, you're losing *both* of these.
The templates are extremely complicated in implementation, which is irrelevant. If templates were rejected based on extremely complicated implementation, that would rule out essentially everything that uses Scribuntu under the hood.
We already have user studies (which I don't have the links for offhand) that
show that people see the relatively simple infobox templates in articles and give up.
Heh. Studies. Complete this phrase: "Lies, damn lies, and..." :-P Seriously, though, I know from direct observation there are people who have trouble with templates comparable to Wikipedian infoboxes; of course, named parameters are a rather baroque aspect of template call syntax, which I've already remarked is not as light-weight as one could wish. (Although in my experience most people don't get it overly wrong, and after being shown once what they'd not quite gotten right, usually don't get it wrong again.) It does seem a bit surprising to suggest that somebody who can't figure out how to fill in the blanks in an infobox would be more able to write in Lua.
The advantage to having the complexity of Lua in modules is that the
complexity specifically is hidden from new users. Once someone gets to the point where they're ready to make that step, they're probably motivated enough that looking through the manual (or even better would be if someone would write some good tutorials) isn't likely to be the barrier it would be to a complete newbie.
Tutorials are a /horrible/ way to convey information. As are manuals. This is one of the key weaknesses of wikis, I think, that they're atrocious at passing on wiki-expertise. Users can learn some things by the incremental path I've described, but beyond that, wikis naturally fall back on documents, because that's all the wiki software is good for specifying. Documents. Afaik, /all/ the sister projects complain that they provide these carefully written welcome templates that explain all about how to do things on that project, and nobody reads them. New variant on RTFM. I once put a welcome template on a newcomer's user talk on en.wb and they actually complained to me about it, saying they'd been to lots of different sister projects and they all annoyingly insisted on /welcoming/ them, they'd been welcomed to death, cut it out already.
What I think /can/ work is an interactive wizard, where at every step the advice/help for that particular step is right there, in-your-face. Hence my intent to provide tools that enable wiki communities to build their own wizards, completely cutting out the middleman (because the wiki community is where the expertise already resides, and wikis thrive on just-do-it, cutting out the middleman).
I don't think most users are going to progress from wiki markup to Lua. Certainly it requires a deeper commitment than the sort of incremental progression I'm advocating. And lowering the bar for participation is also part of what wikis are about.
Sure you can; a little boilerplate is needed, but both that and the
language itself are simple enough that someone could start from some basic examples and experiment easily enough.
Oh, lol. What was that you were saying about cargo-cult programming? (This is meant as /good-natured/ ribbing. I invite you to laugh with me.)
Seriously, cargo-cult programming, or the equivalent, is how much of wiki contribution works (I suppose we shouldn't open the cargo-cult-professional-programming can of worms). Somebody with deeper understanding might come along later and improve things, and that's good too, but that's later.
On the other hand, Lua's syntax and operation is similar to that of C, PHP, JavaScript, and so on, which are likely to be familiar to users with some sort of programming experience. While parser function syntax somewhat resembles Lisp, I'm not sure that similarity will help with understanding more than the surface.
I mean to avoid cutting off the long tail of users who /don't/ have some sort of programming experience. I'm not worried about those whose experience somehow leaves them able to figure out Lua but not minimalist lisp; I'm not sure that's even a nonempty set, and if it is I doubt it's all that significant. Give me your huddled masses yearning to breathe free any day. Also, most of them don't have to understand more than the surface.
I'm not also sure that Lisp's extremely simple syntax is necessarily an
advantage. While extra syntactic structure requires extra learning, it also means that the written code is more structured which may be easier for humans to make sense of. Again, look at the trouble people report in understanding complicated templates.
It's possible for very compact stuff to be hard to understand because it's so compact (APL comes to mind :-). But if it's all in one place (because it's so compact) that also is an advantage. I don't find the comparison with complicated templates compelling, though; seems to me that's heavy syntax.
Of course, my primary goal was interactive pages, for building wizards, and it was clear to me that Lua wasn't going to help me with those
anyway.
I don't see how Lisp is helping that either. If you want interactive pages, JavaScript is the way to go. Or did you write your Lisp parser in JavaScript too?
No, I don't write the Lisp parser in JavaScript. My tools mediate transferring data between different parts of the system, remember. One of the things they can do is snarf the entire raw markup of another wiki page and feed it into the template-expansion process as a template parameter. (No, that's not easy to do, at least I don't know an easy way to do it, and it's the point at which I'm concerned the current implementation may not scale well to large wiki pages.) So the problem of transforming a page --- such as the way HotCat does, or the way the "submit for review" button on a Wikinews article does, or whatever --- is reduced to a string operation. But it's really quite a sophisticated string operation. My embedded specialized lisp dialect has primitives specifically chosen to be good at that sort of thing.
Isn't your Lisp stuff also half-replacing templates with something else? Or are you overlooking that because Lisp syntax resembles that of templates?
I quite agree that my lisp interpreter is not improving templates, but providing in effect an alternative to them in certain cases. It's also an alternative in those cases to Scribuntu --- an alternative that remains embedded in wiki markup.
On 09/02/2014 03:27 PM, pi zero wrote:
The templates are extremely complicated in implementation, which is irrelevant. If templates were rejected based on extremely complicated implementation, that would rule out essentially everything that uses Scribuntu under the hood.
I don't agree with this. Scribunto is a new language to learn (Lua), but that language is far easier for complicated work than parser functions are for complicated work.
Matt Flaschen
Lua makes editing templates for most users impossible. Perhaps the English community has enough users to handle all edit requests for Lua templates, most smaller wikis have too less users to do that. It is not just creating a Lua template, but also that the local communities on various wikis need to be able to maintain those templates when an update is needed or a changed is wished for. Depending on other projects is a bad scenario which is not an acceptable solution.
I also do not see why it is considered easier... it is actually not easier. The general rule which is applied for the wikis I am active is that almost all templates should be editable by the majority of the community and we should not rely on a small number of users with programming skills. And learning an extra method for editing templates is the other way round, which absolutely does not make editing templates easier but creates an extra doorstep for users to do simple updates.
Another general rule applied is KISS: if we can make it simple such is recommended over a complex template with little or no extra functionality. With a complex template, the first question to be asked is if a complex template is really needed. In most cases it is not. Problem solved.
Romaine
2014-10-10 1:40 GMT+02:00 Matthew Flaschen mflaschen@wikimedia.org:
On 09/02/2014 03:27 PM, pi zero wrote:
The templates are extremely complicated in implementation, which is irrelevant. If templates were rejected based on extremely complicated implementation, that would rule out essentially everything that uses Scribuntu under the hood.
I don't agree with this. Scribunto is a new language to learn (Lua), but that language is far easier for complicated work than parser functions are for complicated work.
Matt Flaschen
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/ wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
I have a tool to migrate templates. If anyone has a particular wiki that needs templates migrating on an as needed basis, please let me know and I will be happy to help.
On 10 October 2014 02:57, Romaine Wiki romaine.wiki@gmail.com wrote:
Lua makes editing templates for most users impossible. Perhaps the English community has enough users to handle all edit requests for Lua templates, most smaller wikis have too less users to do that. It is not just creating a Lua template, but also that the local communities on various wikis need to be able to maintain those templates when an update is needed or a changed is wished for. Depending on other projects is a bad scenario which is not an acceptable solution.
I also do not see why it is considered easier... it is actually not easier. The general rule which is applied for the wikis I am active is that almost all templates should be editable by the majority of the community and we should not rely on a small number of users with programming skills. And learning an extra method for editing templates is the other way round, which absolutely does not make editing templates easier but creates an extra doorstep for users to do simple updates.
Another general rule applied is KISS: if we can make it simple such is recommended over a complex template with little or no extra functionality. With a complex template, the first question to be asked is if a complex template is really needed. In most cases it is not. Problem solved.
Romaine
2014-10-10 1:40 GMT+02:00 Matthew Flaschen mflaschen@wikimedia.org:
On 09/02/2014 03:27 PM, pi zero wrote:
The templates are extremely complicated in implementation, which is irrelevant. If templates were rejected based on extremely complicated implementation, that would rule out essentially everything that uses Scribuntu under the hood.
I don't agree with this. Scribunto is a new language to learn (Lua), but that language is far easier for complicated work than parser functions
are
for complicated work.
Matt Flaschen
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/ wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
On 09/02/2014 01:35 PM, pi zero wrote:
(1) It's very easy to use. (2) it naturally promotes incremental learning.
I'm sorry, but both of those assertions are not only wrong, but profoundly misguided.
Wikimarkup, and templates, are /relatively/ easy to use for someone who has at least a passing understanding of markup, and at least some familiarity with common computer science idioms (things like separators, arguments, substitution).
That set of someones used to be a very high fraction of the original batch of editors (who were, almost by necessity, highly-computer litterate geeks). I would be surprised if it represented even a tenth of a percent of today's Internet users.
The typical user, upon seeing {{something|with|stuff}}, will simply turn away saying something along the lines of "I don't know how to program". The only reason templates were a success[1] is because the original wikipedian self-selected by their ability to grok and manipulate those concepts.
-- Marc
[1] Furthermore, even /whether/ templates were a success is highly debatable. If I look at the current mess, and the troubles caused by it, I doubt it. I'd argue that we did great things /despite/ templates as a mechanism, not because of it.
On Tue, Sep 2, 2014 at 2:25 PM, Marc A. Pelletier marc@uberbox.org wrote:
On 09/02/2014 01:35 PM, pi zero wrote:
(1) It's very easy to use. (2) it naturally promotes incremental learning.
I'm sorry, but both of those assertions are not only wrong, but profoundly misguided.
At first I thought, well, seems safe to assume /we're/ not going to agree. And of course I still think that. But it sees there may more going on here than a disagreement about the user experience, in that we may have somewhat different understands of what we're talking about.
I would be surprised if it represented even a tenth
of a percent of today's Internet users.
This is mostly simple disagreement, but may edge into the second point; my first thought was that you're giving people too little credit... but on second thought, I wonder if you're, more specifically, assigning discredit to people that belongs to accidental characteristics of the interface. And then there's this last bit:
The only reason templates were a success[1] is because the original
wikipedian self-selected by their ability to grok and manipulate those concepts.
[1] Furthermore, even /whether/ templates were a success is highly debatable. If I look at the current mess, and the troubles caused by it, I doubt it. I'd argue that we did great things /despite/ templates as a mechanism, not because of it.
Now, just incidentally, besides skepticism on the point about self-selection, I'm also not altogether convinced it /matters/ either way since the result was hugely successful. But what really gave me mental whiplash was the apparent supposition that someone here thinks templates were a success. I'm satisfied we disagree on the manageability of elementary template syntax, but... templates a success? I'm not sure where we're talking past each other, but it's happening somewhere.
On 2 September 2014 17:41, Brad Jorsch (Anomie) bjorsch@wikimedia.org wrote:
<This reply still isn't anything official, but does represent my own views as a developer (both volunteer and staff)>
Speaking generally, rather than about the specific issue at hand, I'm not sure how we can distinguish "something official" from "views as a staff developer".
Am I missing something?
<note this reply is entirely in my personal capacity and in no way represents anything official>
On Tue, Sep 2, 2014 at 5:40 AM, Martijn Hoekstra martijnhoekstra@gmail.com wrote:
- The mobile skin obfuscates talk page access because the templates
commonly found on talk pages makes them render horribly.
- The mobile skin special-cases some templates (notably issue templates and
infoboxes) because they would render horribly.
I may be wrong or out of date on this, but I believe a fair bit of this comes from two issues: 1. Layout templates are sometimes designed with desktop widths in mind, so they sometimes don't respond well to smaller screens. 2. The Mobile site strips inline CSS, in an attempt to overcome item #1.
On a technical level, at least #2 is meant to be addressed by https://www.mediawiki.org/wiki/Requests_for_comment/Allow_styling_in_templat.... Then templates could have associated styles that used proper media queries.
- Media-viewer has a tough time doing to correct thing with attribution
and license information because parsing template-madness is hard.
IMO, the "structured data for Commons" initiative should have come before MediaViewer tried to incorporate attribution and licensing, rather than trying to fake it with parsing templates.
In defense of the MV folks, though, it seems that decision was made when MV had a much smaller scope for the information it would include.
- VE development has spent a large amount of time around templates, and
it's still one of its weakest suits. Template substitution is still a problem, as well as templates that produce wikitext that in itself doesn't map cleanly to HTML tokens.
I don't think there's any way around this one, VE just needs to improve.
As much as some might like the idea of not having templates like {{hidden top}} and {{hidden bottom}}, the alternative of passing a huge wad of content into {{hidden}} doesn't seem like it's always workable.
* Scribunto has been developed specifically because writing and maintaining
templates with more complicated logic is horrible, both from a writers/maintainers perspective as well as from a performance perspective
Scribunto isn't a problem with templates, it's the solution ;)
AFAIK, templates were originally designed as just that: plug a few variables into a structure.
But then you realize that sometimes you want to be able to output bits of surrounding structure only if the variable is non-empty, or you want to pad or truncate a value, or you need to use the same variable value in a different case or percent-encoded, or you realize that it's easier to embed {{#switch:{{{var}}}|...}} instead of having 100 subtemplates and transcluding one with {{example/{{{var}}}}} or having a tree of {{#if:}}, or you need to deal with formatting times and dates, so you add parser functions.
And then people use those parser functions to start actually *programming* in templates (like taking advantage of the padding/truncation function to write really awesome/awful implementations of strlen, substr, and so on) to make the user interface of the template easier to use. But doing this programming in a language designed only for templating doesn't lead to anything easy to write or read.
Scribunto answers this by letting people program in a language designed for programming rather than hacking things up in a language designed for templating. And further, Scribunto specifically doesn't support embedding the programming code inside the templates, as allowing that would likely make readability worse rather than better. (Yes, I know some people have done crazy things like writing parsers in Lua so they can embed code in template arguments. While it's a cool hack, I think actually using it would be misguided.)
I agree with the general thrust of Brad's points. There are some known major issues with templating (styling, unreadable parser functions, and templates being used for data like Information on Commons are the most important ones).
However, these do not all have the same solutions. Some issues are in a bit stalled but the idea is known (e.g. https://www.mediawiki.org/wiki/Requests_for_comment/Allow_styling_in_templat...). Others have an understood solution (e.g. Wikidata for meta-data of the sort used on Commons).
Finally, I consider the "unreadable parser functions" problem essentially solved. Lua is not perfect, but it's a usable language (and not a Mediawiki-specific one) that is far more readable and writable than complicated nested parser functions.
I think we should standardize on JavaScript (for interactivity) and Scribunto (for business logic that supports templates) as the on-wiki languages for non-trivial logic, leaving the wikitext template side for simple presentation.
Matt Flaschen
On Thu, Oct 9, 2014 at 7:53 PM, Matthew Flaschen mflaschen@wikimedia.org wrote:
Finally, I consider the "unreadable parser functions" problem essentially solved. Lua is not perfect, but it's a usable language (and not a Mediawiki-specific one) that is far more readable and writable than complicated nested parser functions.
I see replacing "unreadable parser functions" with "Lua" as replacing one bad solution with a differently bad solution. Since the problems of the second are different from those of the first, it's possible to claim the second "solves the problems" of the first, but that's a bit lame. The problems of Lua should have been anticipated, and should have been avoided by finding a better solution; and settling for a differently bad solution is just a different way of settling for a bad solution.
Martijn,
First, *thank you* for drafting this -- I believe this is a fundamental architectural and maintenance issue we need to resolve. It is also a long term innovation issue -- as it is one of the ways individuals can participate in extending our user experience.
This would be a great case to plan and work on together, starting with gathering all of the different use cases currently covered, goals, problems, catalogue of current templates with usage data and desired improvements via a requirement document on-wiki.
We (WMF engineering) should provide support for this (so this complements any refactoring in Media Wikis and features under development), which means we would need to prioritize and schedule on our side as well. But requirement gathering can start now.
Martijn -- thank you for this, Lila
On Tue, Sep 2, 2014 at 2:40 AM, Martijn Hoekstra martijnhoekstra@gmail.com wrote:
tl;dr: We've been collectively whining about templates for long enough. Who wants to help with fixing them?
In the recent discussions/debacles about technical and stylistic advances, a recurring theme is that the use of some templates causes major headaches, and a commonly heard complaint from the developers and designers is that their products exhibit problems and shortcoming because of that. Anecdotal evidence I've lately encountered includes:
- The mobile skin obfuscates talk page access because the templates
commonly found on talk pages makes them render horribly.
- The mobile skin special-cases some templates (notably issue templates and
infoboxes) because they would render horribly.
- Media-viewer has a tough time doing to correct thing with attribution and
license information because parsing template-madness is hard.
- VE development has spent a large amount of time around templates, and
it's still one of its weakest suits. Template substitution is still a problem, as well as templates that produce wikitext that in itself doesn't map cleanly to HTML tokens.
- Scribunto has been developed specifically because writing and maintaining
templates with more complicated logic is horrible, both from a writers/maintainers perspective as well as from a performance perspective
All this together is sufficient to assert we have a template problem. The main editing community has a problem with how templates are and must be used, the readers have a problem with display issues on mobile as well as style inconsistencies, the technical editing community has a problem with writing and maintaining templates, and the development community has a problem with the difficulty in correctly parsing and interpreting templates and there contents.
It would be great if this problem were tackled; it would be even greater if the WMF could work together with the community to identify the pain points, and jointly take steps to tackle them. Templates are currently extraordinarily powerful, and most if not all of this power is finding use in the projects, possibly in ways nobody ever foresaw. As we all know from Uncle Ben, with great power comes great responsibility, and it's about time we all took our share of that responsibility, tough up, and fix it.
We should keep in mind that current use is paramount, and any fixing of templates that breaks the wiki is frankly unacceptable, which probably means we can't go from insane to sane overnight, even if we could define sane and insane with regards to templates overnight. At the same time we shouldn't shy away from fixes that would break some exotic use of templates, if as part of the process of making things better, before implementation, we can fix those templates.
I hope we can, for the coming period, accomplish the following:
- Catalog the problems with templates. Make a comprehensive list that
enumerates the problems with templates we have now, categories the problems (right now I'm roughly thinking in style, wikitext parsing rules and generated HTML, creation and writing issues (let's hope there is little of this one left after Scribunto), and usability by editors).
- Note which quirks that lead to technical difficulties are used in the
wild as features rather than bugs.
- Brain storm possible (partial) solutions.
- Find candidates that have high bang-for-buck possible solutions without
impeding future improvements much.
- Refine those solutions so we know quite exactly what it will fix, what it
won't fix, and what it would possibly break.
- Define sane fallback procedures for when things break; this should mainly
come from the editing communities, but could probably use some guidance of what is possible/easy/logical/feasible from a technical POV from the development community.
- Fix templates.
Personally, I'm all talk and no action, so to get this of the ground we would need a lot of help. First, we need to know if I'm on to something, or if this is just the raving of a lunatic (please tell me if it is!). If the idea is sound, we need to set up the infrastructure. We probably need a Meta page set up to organise things and set up initial reconnaissance. We need a lot of grunt work categorising issues and problems from all perspectives: reader (this is difficult, but many groups that don't directly represent the readers care deeply about their needs, so that's something), template users, template maintainers, and template infrastructure maintainers (developers). For that we need to reach out to those different communities; this email is posted to wikimedia-l only (because I couldn't think of a better one, but I acknowledge this doesn't fit like a glove), but there are bound to be other interested parties out there who want to help that this email isn't reaching.
What do you all think? Should we make this happen?
--Martijn _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Hi Martijn. Thanks for starting this thread.
Martijn Hoekstra [roughly] wrote:
- Catalog the problems with [dev issue]. Make a comprehensive list that
enumerates the problems with [dev issue] we have now, categorise the problems (right now I'm roughly thinking in style, wikitext parsing rules and generated HTML, creation and writing issues (let's hope there is little of this one left after Scribunto), and usability by editors).
- Note which quirks that lead to technical difficulties are used in the
wild as features rather than bugs.
- Brainstorm possible (partial) solutions.
- Find candidates that have high bang-for-buck possible solutions
without impeding future improvements much.
- Refine those solutions so we know quite exactly what it will fix, what
it won't fix, and what it would possibly break.
- Define sane fallback procedures for when things break; this should
mainly come from the editing communities, but could probably use some guidance of what is possible/easy/logical/feasible from a technical POV from the development community.
- Fix [dev issue].
I don't disagree with what you're saying, but I don't think it's really specific to templates. We should roughly be following these steps for most development issues, no?
There won't be a "one size fits all" approach to templates.
In the recent discussions/debacles about technical and stylistic advances, a recurring theme is that the use of some templates causes major headaches, and a commonly heard complaint from the developers and designers is that their products exhibit problems and shortcoming because of that. Anecdotal evidence I've lately encountered includes:
- The mobile skin obfuscates talk page access because the templates
commonly found on talk pages makes them render horribly.
Talk page templates are dumb. We should integrate their functionality into a MediaWiki extension. We currently have people going around assessing articles on talk pages and adding talk page banners using iterator tools such as AutoWikiBrowser. These are fine people and they're doing fine work, but the mechanism by which they're doing it is sorely lacking. We can easily store and manage page properties such as an article's quality assessment or which WikiProjects are interested in the article in a structured database. We already track (and can query!) by many page properties. Let's leverage the software platform we've built.
Regardless of whether we use a built-in tool or we continue to use templates, you're talking about trashing templates because of CSS issues. That doesn't make much sense to me. Templates make life vastly easier. We can likely update most talk page templates on large wikis with a single edit to a meta-template or perhaps even just a CSS edit. That's great!
- The mobile skin special-cases some templates (notably issue templates
and infoboxes) because they would render horribly.
Second mention of the mobile skin. Perhaps it's the mobile skin that needs work. I think the mobile experience is horribly and painfully deficient and flawed. Any help you can provide in trying to make it better would surely be welcome. I've tried a few times and it only results in sadness.
- Media-viewer has a tough time doing to correct thing with attribution
and license information because parsing template-madness is hard.
Structured data, a.k.a. Wikidata, as Brad noted.
- VE development has spent a large amount of time around templates, and
it's still one of its weakest suits. Template substitution is still a problem, as well as templates that produce wikitext that in itself doesn't map cleanly to HTML tokens.
VisualEditor actually doesn't deal with wikitext, as I understand it, it deals only with HTML. The Parsoid team deals with wikitext and they knew what they were getting into long before this project started. There was a commitment made to support most wikitext. And so it goes.
And as Brad notes, both VisualEditor and Parsoid will have to deal with annoying use-cases. (Of all the features, I think {{hidden top}}-type functionality will probably be one of the easier features to implement.)
- Scribunto has been developed specifically because writing and
maintaining templates with more complicated logic is horrible, both from a writers/maintainers perspective as well as from a performance perspective
This isn't really an argument against templates. Scribunto is part of the solution, not part of the problem. In theory, anyway.
All this together is sufficient to assert we have a template problem.
I think there's a reasonable case to be made against specific uses of templates (talk page templates are an easy target, as discussed above).
The steps you laid out here make sense and seem reasonable, but you'll have to do a new pass-through of these steps for each general template use-case you discover and wish to address. And each pass-through takes close to a year in my experience, if not longer. And that's with dedicated development resources.
One point that you didn't hit on which is an even easier target for criticism is that there's currently no easy way of sharing templates between wikis (no interwiki templates). We need to shrink the overall number of templates by finding common patterns between wikis and smartly sharing code _and_ man-power. If you want to make a substantive impact on the number of templates, I'd strongly suggest figuring out a way to revive interwiki transclusion or an equivalent sharing mechanism.
MZMcBride
On Sep 3, 2014 4:46 AM, "MZMcBride" z@mzmcbride.com wrote:
Hi Martijn. Thanks for starting this thread.
Martijn Hoekstra [roughly] wrote:
- Catalog the problems with [dev issue]. Make a comprehensive list that
enumerates the problems with [dev issue] we have now, categorise the problems (right now I'm roughly thinking in style, wikitext parsing rules and generated HTML, creation and writing issues (let's hope there is little of this one left after Scribunto), and usability by editors).
- Note which quirks that lead to technical difficulties are used in the
wild as features rather than bugs.
- Brainstorm possible (partial) solutions.
- Find candidates that have high bang-for-buck possible solutions
without impeding future improvements much.
- Refine those solutions so we know quite exactly what it will fix, what
it won't fix, and what it would possibly break.
- Define sane fallback procedures for when things break; this should
mainly come from the editing communities, but could probably use some guidance of what is possible/easy/logical/feasible from a technical POV from the development community.
- Fix [dev issue].
Yeah there are a couple of dev issues above. This email is purposefully sent to the broadest list I could find, because it are broad issues, and the dev community is part of the audience of this list. One of the issues (but not the only issue) is that if templates were less troublesome, it would be easier for dev to make great products. It would be easy to say that is their problem, but since everyone benefits from the software being better, helping dev by reducing template madness helps all of us.
I don't disagree with what you're saying, but I don't think it's really specific to templates. We should roughly be following these steps for most development issues, no?
There won't be a "one size fits all" approach to templates.
In the recent discussions/debacles about technical and stylistic
advances,
a recurring theme is that the use of some templates causes major headaches, and a commonly heard complaint from the developers and designers is that their products exhibit problems and shortcoming because of that. Anecdotal evidence I've lately encountered includes:
- The mobile skin obfuscates talk page access because the templates
commonly found on talk pages makes them render horribly.
Talk page templates are dumb. We should integrate their functionality into a MediaWiki extension. We currently have people going around assessing articles on talk pages and adding talk page banners using iterator tools such as AutoWikiBrowser. These are fine people and they're doing fine work, but the mechanism by which they're doing it is sorely lacking. We can easily store and manage page properties such as an article's quality assessment or which WikiProjects are interested in the article in a structured database. We already track (and can query!) by many page properties. Let's leverage the software platform we've built.
Regardless of whether we use a built-in tool or we continue to use templates, you're talking about trashing templates because of CSS issues. That doesn't make much sense to me. Templates make life vastly easier. We can likely update most talk page templates on large wikis with a single edit to a meta-template or perhaps even just a CSS edit. That's great!
- The mobile skin special-cases some templates (notably issue templates
and infoboxes) because they would render horribly.
Second mention of the mobile skin. Perhaps it's the mobile skin that needs work. I think the mobile experience is horribly and painfully deficient and flawed. Any help you can provide in trying to make it better would surely be welcome. I've tried a few times and it only results in sadness.
There are a couple of interlocking problems here. Templates are often inline styled for the desktop. Writing and maintaining inline styles is a bit cumbersome, writing good styles that work for desktop and mobile isn't all that easy, and those two amplify each other.
The mobile skin does the wrong thing (stripping inline styles) because the templates have inline styles that aren't suitable for mobile, and they have to do something to get an acceptable end result.
The templates don't have suitable styles for mobile because it's so hard to inline style something. If the styling of the templates becomes less cumbersome, we can start making them better suited for mobile, the mobile skin can stop special casing them, and everyone will be happier.
- Media-viewer has a tough time doing to correct thing with attribution
and license information because parsing template-madness is hard.
Structured data, a.k.a. Wikidata, as Brad noted.
- VE development has spent a large amount of time around templates, and
it's still one of its weakest suits. Template substitution is still a problem, as well as templates that produce wikitext that in itself doesn't map cleanly to HTML tokens.
VisualEditor actually doesn't deal with wikitext, as I understand it, it deals only with HTML. The Parsoid team deals with wikitext and they knew what they were getting into long before this project started. There was a commitment made to support most wikitext. And so it goes.
And as Brad notes, both VisualEditor and Parsoid will have to deal with annoying use-cases. (Of all the features, I think {{hidden top}}-type functionality will probably be one of the easier features to implement.)
Part of the community - the part that makes parsoid/ve - has a problem with some template uses. Ignoring the
- Scribunto has been developed specifically because writing and
maintaining templates with more complicated logic is horrible, both from a writers/maintainers perspective as well as from a performance perspective
This isn't really an argument against templates. Scribunto is part of the solution, not part of the problem. In theory, anyway.
In practice too, but I think that with Scribunto enabled we can maybe start thinking about deprecating (some) parser functions, or at least throw that possibility on the table.
All this together is sufficient to assert we have a template problem.
I think there's a reasonable case to be made against specific uses of templates (talk page templates are an easy target, as discussed above).
The steps you laid out here make sense and seem reasonable, but you'll have to do a new pass-through of these steps for each general template use-case you discover and wish to address. And each pass-through takes close to a year in my experience, if not longer. And that's with dedicated development resources.
Having a holistic solution would be nice, but starting incrementally seems viable. Also, fortunately, there are (still?) lots of us. If we can find a way to streamline and harness even a fraction of the brain- and willpower of the collective Wikimedia movement, we're golden. That's obviously a big if though - I definitely agree I can't go this alone, or that the WMF can go this alone (for various reasons) for example.
One point that you didn't hit on which is an even easier target for criticism is that there's currently no easy way of sharing templates between wikis (no interwiki templates). We need to shrink the overall number of templates by finding common patterns between wikis and smartly sharing code _and_ man-power. If you want to make a substantive impact on the number of templates, I'd strongly suggest figuring out a way to revive interwiki transclusion or an equivalent sharing mechanism.
Great point. iirc cross wiki Lua modules is the most popular requested Scribunto feature. The same could go for templates, though if we bring templates back to simply putting a few words in a (localised) text template and doing everything else in cross wiki Scribunto modules, it might not even be needed. I think that's the right(tm) way to use templates, but I'm not sure if we can ever give up completely the template power we have at our disposal now; they're versatile and very heavily used for some pretty awesome things. This is definitely in scope of lets improve templates though, so thank you for bringing it up.
--Martijn
tl;dr: We've been collectively whining about templates for long enough. Who wants to help with fixing them?
I want to help fix them.
I hope we can, for the coming period, accomplish the following:
- Catalog the problems with templates. Make a comprehensive list that
enumerates the problems with templates we have now, categories the problems (right now I'm roughly thinking in style, wikitext parsing rules and generated HTML, creation and writing issues (let's hope there is little of this one left after Scribunto), and usability by editors).
- Note which quirks that lead to technical difficulties are used in the
wild as features rather than bugs.
- Brain storm possible (partial) solutions.
- Find candidates that have high bang-for-buck possible solutions without
impeding future improvements much.
- Refine those solutions so we know quite exactly what it will fix, what it
won't fix, and what it would possibly break.
- Define sane fallback procedures for when things break; this should mainly
come from the editing communities, but could probably use some guidance of what is possible/easy/logical/feasible from a technical POV from the development community.
- Fix templates.
I'd like to add distribution as one of the pain points. I wanted to have the templates that are available on enwiki for another Mediawiki installation, but I couldn't get them to work. It seems like every template has a maze of dependencies, and when I resorted to exporting all of the templates from the Mediawiki site, the software consistently crashed before all templates were exported. I might have been doing it wrong, but I couldn't find any other options. Ideally, something like a package management system for templates, extensions, and skins would be a godsend.
What do you all think? Should we make this happen?
I'm no template expert, but I agree with a lot of what you've said based on the experiences I've had. I think we should discuss this somewhere that is less transient than this list. ln short, I'm down.
,Wil
On Wed, Sep 3, 2014 at 10:04 AM, Wil Sinclair wllm@wllm.com wrote:
tl;dr: We've been collectively whining about templates for long enough.
Who
wants to help with fixing them?
I want to help fix them.
Great to hear. Getting my ass in to gear is one of my greatest weaknesses, and from what I know from you you're really good at that. :)
I hope we can, for the coming period, accomplish the following:
- Catalog the problems with templates. Make a comprehensive list that
enumerates the problems with templates we have now, categories the
problems
(right now I'm roughly thinking in style, wikitext parsing rules and generated HTML, creation and writing issues (let's hope there is little
of
this one left after Scribunto), and usability by editors).
- Note which quirks that lead to technical difficulties are used in the
wild as features rather than bugs.
- Brain storm possible (partial) solutions.
- Find candidates that have high bang-for-buck possible solutions without
impeding future improvements much.
- Refine those solutions so we know quite exactly what it will fix, what
it
won't fix, and what it would possibly break.
- Define sane fallback procedures for when things break; this should
mainly
come from the editing communities, but could probably use some guidance
of
what is possible/easy/logical/feasible from a technical POV from the development community.
- Fix templates.
I'd like to add distribution as one of the pain points. I wanted to have the templates that are available on enwiki for another Mediawiki installation, but I couldn't get them to work. It seems like every template has a maze of dependencies, and when I resorted to exporting all of the templates from the Mediawiki site, the software consistently crashed before all templates were exported. I might have been doing it wrong, but I couldn't find any other options. Ideally, something like a package management system for templates, extensions, and skins would be a godsend.
A "Mediawiki core templates pack" sounds like a good idea - as would making template and module interdependencies explicit to somewhat avoid the Great Tangle.
aside - wikitech-l as well as #mediawiki freenode are in my experience happy to help with individual setup woes, which could help you with the acute import issues /aside
What do you all think? Should we make this happen?
I'm no template expert, but I agree with a lot of what you've said based on the experiences I've had. I think we should discuss this somewhere that is less transient than this list.
Stop rushing me all y'all! ;)
ln short, I'm down.
,Wil
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Exsqueeze the ignorance. I'm still a n00b. Martijn, where should we set this discussion up?
It's clear that there are several people who are interested in talking templates. I'm getting my hands dirty with them on another project I'm working on. I don't mean to rush you; just tell me what to set up for this discussion and where, and I'll make sure it gets done.
,Wil
On Wed, Sep 3, 2014 at 1:48 AM, Martijn Hoekstra martijnhoekstra@gmail.com wrote:
On Wed, Sep 3, 2014 at 10:04 AM, Wil Sinclair wllm@wllm.com wrote:
tl;dr: We've been collectively whining about templates for long enough.
Who
wants to help with fixing them?
I want to help fix them.
Great to hear. Getting my ass in to gear is one of my greatest weaknesses, and from what I know from you you're really good at that. :)
I hope we can, for the coming period, accomplish the following:
- Catalog the problems with templates. Make a comprehensive list that
enumerates the problems with templates we have now, categories the
problems
(right now I'm roughly thinking in style, wikitext parsing rules and generated HTML, creation and writing issues (let's hope there is little
of
this one left after Scribunto), and usability by editors).
- Note which quirks that lead to technical difficulties are used in the
wild as features rather than bugs.
- Brain storm possible (partial) solutions.
- Find candidates that have high bang-for-buck possible solutions without
impeding future improvements much.
- Refine those solutions so we know quite exactly what it will fix, what
it
won't fix, and what it would possibly break.
- Define sane fallback procedures for when things break; this should
mainly
come from the editing communities, but could probably use some guidance
of
what is possible/easy/logical/feasible from a technical POV from the development community.
- Fix templates.
I'd like to add distribution as one of the pain points. I wanted to have the templates that are available on enwiki for another Mediawiki installation, but I couldn't get them to work. It seems like every template has a maze of dependencies, and when I resorted to exporting all of the templates from the Mediawiki site, the software consistently crashed before all templates were exported. I might have been doing it wrong, but I couldn't find any other options. Ideally, something like a package management system for templates, extensions, and skins would be a godsend.
A "Mediawiki core templates pack" sounds like a good idea - as would making template and module interdependencies explicit to somewhat avoid the Great Tangle.
aside - wikitech-l as well as #mediawiki freenode are in my experience happy to help with individual setup woes, which could help you with the acute import issues /aside
What do you all think? Should we make this happen?
I'm no template expert, but I agree with a lot of what you've said based on the experiences I've had. I think we should discuss this somewhere that is less transient than this list.
Stop rushing me all y'all! ;)
ln short, I'm down.
,Wil
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
I'm thinking a page on Meta to start with. I'll get round to it when I get home from work tonight CET.
On Wed, Sep 3, 2014 at 11:33 AM, Wil Sinclair wllm@wllm.com wrote:
Exsqueeze the ignorance. I'm still a n00b. Martijn, where should we set this discussion up?
It's clear that there are several people who are interested in talking templates. I'm getting my hands dirty with them on another project I'm working on. I don't mean to rush you; just tell me what to set up for this discussion and where, and I'll make sure it gets done.
,Wil
On Wed, Sep 3, 2014 at 1:48 AM, Martijn Hoekstra martijnhoekstra@gmail.com wrote:
On Wed, Sep 3, 2014 at 10:04 AM, Wil Sinclair wllm@wllm.com wrote:
tl;dr: We've been collectively whining about templates for long
enough.
Who
wants to help with fixing them?
I want to help fix them.
Great to hear. Getting my ass in to gear is one of my greatest
weaknesses,
and from what I know from you you're really good at that. :)
I hope we can, for the coming period, accomplish the following:
- Catalog the problems with templates. Make a comprehensive list that
enumerates the problems with templates we have now, categories the
problems
(right now I'm roughly thinking in style, wikitext parsing rules and generated HTML, creation and writing issues (let's hope there is
little
of
this one left after Scribunto), and usability by editors).
- Note which quirks that lead to technical difficulties are used in
the
wild as features rather than bugs.
- Brain storm possible (partial) solutions.
- Find candidates that have high bang-for-buck possible solutions
without
impeding future improvements much.
- Refine those solutions so we know quite exactly what it will fix,
what
it
won't fix, and what it would possibly break.
- Define sane fallback procedures for when things break; this should
mainly
come from the editing communities, but could probably use some
guidance
of
what is possible/easy/logical/feasible from a technical POV from the development community.
- Fix templates.
I'd like to add distribution as one of the pain points. I wanted to have the templates that are available on enwiki for another Mediawiki installation, but I couldn't get them to work. It seems like every template has a maze of dependencies, and when I resorted to exporting all of the templates from the Mediawiki site, the software consistently crashed before all templates were exported. I might have been doing it wrong, but I couldn't find any other options. Ideally, something like a package management system for templates, extensions, and skins would be a godsend.
A "Mediawiki core templates pack" sounds like a good idea - as would
making
template and module interdependencies explicit to somewhat avoid the
Great
Tangle.
aside - wikitech-l as well as #mediawiki freenode are in my experience happy to help with individual setup woes, which could help you with the acute import issues /aside
What do you all think? Should we make this happen?
I'm no template expert, but I agree with a lot of what you've said based on the experiences I've had. I think we should discuss this somewhere that is less transient than this list.
Stop rushing me all y'all! ;)
ln short, I'm down.
,Wil
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wikimedia-l mailing list, guidelines at:
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Bumping a great thread: Is this the right page?
https://meta.wikimedia.org/wiki/Fixing_templates
On Wed, Sep 3, 2014 at 5:45 AM, Martijn Hoekstra martijnhoekstra@gmail.com wrote:
I'm thinking a page on Meta to start with. I'll get round to it when I get home from work tonight CET.
On Wed, Sep 3, 2014 at 11:33 AM, Wil Sinclair wllm@wllm.com wrote:
Exsqueeze the ignorance. I'm still a n00b. Martijn, where should we set this discussion up?
It's clear that there are several people who are interested in talking templates. I'm getting my hands dirty with them on another project I'm working on. I don't mean to rush you; just tell me what to set up for this discussion and where, and I'll make sure it gets done.
,Wil
On Wed, Sep 3, 2014 at 1:48 AM, Martijn Hoekstra martijnhoekstra@gmail.com wrote:
On Wed, Sep 3, 2014 at 10:04 AM, Wil Sinclair wllm@wllm.com wrote:
tl;dr: We've been collectively whining about templates for long
enough.
Who
wants to help with fixing them?
I want to help fix them.
Great to hear. Getting my ass in to gear is one of my greatest
weaknesses,
and from what I know from you you're really good at that. :)
I hope we can, for the coming period, accomplish the following:
- Catalog the problems with templates. Make a comprehensive list that
enumerates the problems with templates we have now, categories the
problems
(right now I'm roughly thinking in style, wikitext parsing rules and generated HTML, creation and writing issues (let's hope there is
little
of
this one left after Scribunto), and usability by editors).
- Note which quirks that lead to technical difficulties are used in
the
wild as features rather than bugs.
- Brain storm possible (partial) solutions.
- Find candidates that have high bang-for-buck possible solutions
without
impeding future improvements much.
- Refine those solutions so we know quite exactly what it will fix,
what
it
won't fix, and what it would possibly break.
- Define sane fallback procedures for when things break; this should
mainly
come from the editing communities, but could probably use some
guidance
of
what is possible/easy/logical/feasible from a technical POV from the development community.
- Fix templates.
I'd like to add distribution as one of the pain points. I wanted to have the templates that are available on enwiki for another Mediawiki installation, but I couldn't get them to work. It seems like every template has a maze of dependencies, and when I resorted to exporting all of the templates from the Mediawiki site, the software consistently crashed before all templates were exported. I might have been doing it wrong, but I couldn't find any other options. Ideally, something like a package management system for templates, extensions, and skins would be a godsend.
A "Mediawiki core templates pack" sounds like a good idea - as would
making
template and module interdependencies explicit to somewhat avoid the
Great
Tangle.
aside - wikitech-l as well as #mediawiki freenode are in my experience happy to help with individual setup woes, which could help you with the acute import issues /aside
What do you all think? Should we make this happen?
I'm no template expert, but I agree with a lot of what you've said based on the experiences I've had. I think we should discuss this somewhere that is less transient than this list.
Stop rushing me all y'all! ;)
ln short, I'm down.
,Wil
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wikimedia-l mailing list, guidelines at:
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
On Wed, Sep 3, 2014 at 6:04 PM, Wil Sinclair wllm@wllm.com wrote:
I'd like to add distribution as one of the pain points. I wanted to have the templates that are available on enwiki for another Mediawiki installation, but I couldn't get them to work. It seems like every template has a maze of dependencies, and when I resorted to exporting all of the templates from the Mediawiki site, the software consistently crashed before all templates were exported.
This sounds like a bug. Does anyone know of related bugs in bugzilla?
I might have been doing it wrong, but I couldn't find any other options.
One option is to use a tool (e.g. pywikibot) to transfer each template.
https://www.mediawiki.org/wiki/Manual:Pywikibot
This script transferbot.py does some of the required work, but it would be nice if it was improved to detect and copy any dependencies semi-automatically.
https://git.wikimedia.org/blob/pywikibot%2Fcore.git/HEAD/scripts%2Ftransferb...
Ideally, something like a package management system for templates,
+1
.. extensions, and skins would be a godsend.
This seems to be unrelated to templates, as they rarely depend on extensions or skins, and deploy of templates is a very different mechanism to deploy of extensions and skins. fwiw, package management systems do include separate packages for commonly needed extensions and skins.
(Wouldn't it be nice if all skins could be written in Lua + LESS stored as wikipages on the wiki, instead of distributed as php files; I know of one LESS skin which works well: https://www.mediawiki.org/wiki/Skin:Chameleon ; are there others?)
-- John Vandenberg
On 09/03/2014 07:08 AM, John Mark Vandenberg wrote:
(Wouldn't it be nice if all skins could be written in Lua + LESS stored as wikipages on the wiki, instead of distributed as php files; I know of one LESS skin which works well: https://www.mediawiki.org/wiki/Skin:Chameleon ; are there others?)
Vector?
It's now all LESS except the jQuery UI styling.
Matt Flaschen
wikimedia-l@lists.wikimedia.org