I'm using more and more #switch into templates, it's surprising how many issues it can solve, and how much large arrays it can manage. My questions are: 1. Is there a reasonable upper limit for the number of #switch options? 2. Is there a difference in server load between a long list of #switch options and a "tree" of options t.i. something as a nested #switch? 3. Is there any drawback into a large use of #swtch-based templates?
I presume that my questions are far from original, please give me a link to previous talks or help/doc pages if any.
Alex
On Mon, Jan 30, 2012 at 3:11 PM, Alex Brollo alex.brollo@gmail.com wrote:
I'm using more and more #switch into templates, it's surprising how many issues it can solve, and how much large arrays it can manage. My questions are:
- Is there a reasonable upper limit for the number of #switch options?
- Is there a difference in server load between a long list of #switch
options and a "tree" of options t.i. something as a nested #switch? 3. Is there any drawback into a large use of #swtch-based templates?
I can at least tell you that the answer to #1 is probably no, and the answer to #3 is yes. There were some server issues last week that were blamed on ocwiki's use of templates with #switch statements with thousands of options. For instance, https://oc.wikipedia.org/w/index.php?title=Mod%C3%A8l:Altmejcom&action=e... has 36,611 cases, because it's essentially used as a database. That's the kind of thing that takes forever to parse.
Roan
What about ptwikibooks' usage of https://pt.wikibooks.org/wiki/Template:Lista_de_cap%C3%ADtulos/Posterior?act... https://pt.wikibooks.org/wiki/Template:Lista_de_cap%C3%ADtulos/Anterior?acti... https://pt.wikibooks.org/wiki/Template:Lista_de_cap%C3%ADtulos/Imprimir?acti... https://pt.wikibooks.org/wiki/Template:Lista_de_cap%C3%ADtulos/Cole%C3%A7%C3... etc...? Is that acceptable? Those templates were created as a workaround for features which are (still) missing on non-Wikipedia projects, such as these: https://bugzilla.wikimedia.org/showdependencytree.cgi?id=15071&hide_reso...
It would be really great if on this GSoC 2012, https://www.mediawiki.org/wiki/Summer_of_Code_2012 , some of the projects were focused on improving Wikibooks/Wikisources... I hope someone gets interested.
On Mon, Jan 30, 2012 at 15:24, Roan Kattouw roan.kattouw@gmail.com wrote:
On Mon, Jan 30, 2012 at 3:11 PM, Alex Brollo alex.brollo@gmail.com wrote:
I'm using more and more #switch into templates, it's surprising how many issues it can solve, and how much large arrays it can manage. My
questions
are:
- Is there a reasonable upper limit for the number of #switch options?
- Is there a difference in server load between a long list of #switch
options and a "tree" of options t.i. something as a nested #switch? 3. Is there any drawback into a large use of #swtch-based templates?
I can at least tell you that the answer to #1 is probably no, and the answer to #3 is yes. There were some server issues last week that were blamed on ocwiki's use of templates with #switch statements with thousands of options. For instance, https://oc.wikipedia.org/w/index.php?title=Mod%C3%A8l:Altmejcom&action=e... has 36,611 cases, because it's essentially used as a database. That's the kind of thing that takes forever to parse.
Roan
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Mon, Jan 30, 2012 at 8:13 PM, Helder helder.wiki@gmail.com wrote:
What about ptwikibooks' usage of https://pt.wikibooks.org/wiki/Template:Lista_de_cap%C3%ADtulos/Posterior?act...
It looks like that finds the current page name in a list of pages, then returns the next one in the list. I can see how that makes sense for a navigation template, I guess. But does this really need to support up to a thousand (!) parameters? Aren't these pages typically numbered?
Roan
On Mon, Jan 30, 2012 at 17:20, Roan Kattouw roan.kattouw@gmail.com wrote:
On Mon, Jan 30, 2012 at 8:13 PM, Helder helder.wiki@gmail.com wrote:
What about ptwikibooks' usage of
https://pt.wikibooks.org/wiki/Template:Lista_de_cap%C3%ADtulos/Posterior?act... It looks like that finds the current page name in a list of pages, then returns the next one in the list. I can see how that makes sense for a navigation template, I guess.
That is correct. This template is used to generate links to the "next" chapter of the books. See e.g. the blue bars on top and bottom of this chapter: https://pt.wikibooks.org/wiki/GIMP/Caixa_de_ferramentas
But does this really need to support up to a thousand (!) parameters?
Well, I remember at least two books which have more than 500 chapters: https://pt.wikibooks.org/wiki/Template:Lista_de_cap%C3%ADtulos/Log%C3%ADstic... https://pt.wikibooks.org/wiki/Template:Lista_de_cap%C3%ADtulos/Guia_do_Linux and our cookbook has ~2000 chapters: https://pt.wikibooks.org/wiki/Template:Lista_de_cap%C3%ADtulos/Livro_de_rece...
Aren't these pages typically numbered?
Not really. The chapter names are usually "descriptive names" (see examples in the lists above), not just numbers.
Best regards, Helder
On 31/01/12 04:24, Roan Kattouw wrote:
On Mon, Jan 30, 2012 at 3:11 PM, Alex Brollo alex.brollo@gmail.com wrote:
I'm using more and more #switch into templates, it's surprising how many issues it can solve, and how much large arrays it can manage. My questions are:
- Is there a reasonable upper limit for the number of #switch options?
- Is there a difference in server load between a long list of #switch
options and a "tree" of options t.i. something as a nested #switch? 3. Is there any drawback into a large use of #swtch-based templates?
I can at least tell you that the answer to #1 is probably no, and the answer to #3 is yes. There were some server issues last week that were blamed on ocwiki's use of templates with #switch statements with thousands of options. For instance, https://oc.wikipedia.org/w/index.php?title=Mod%C3%A8l:Altmejcom&action=e... has 36,611 cases, because it's essentially used as a database. That's the kind of thing that takes forever to parse.
Actually it's relatively fast to parse, but it uses hundreds of megabytes of unaccounted memory and can even send the servers into swap.
I'm considering introducing a limit on #switch cases of 2000 or so per article, to address this issue. No doubt many templates will break, but it's important to protect our servers, and we've always discouraged this kind of #switch application.
-- Tim Starling
On Mon, Jan 30, 2012 at 3:23 PM, Tim Starling tstarling@wikimedia.orgwrote:
I'm considering introducing a limit on #switch cases of 2000 or so per article, to address this issue. No doubt many templates will break, but it's important to protect our servers, and we've always discouraged this kind of #switch application.
As a Wikipedian, I would be very happy to see a limitation imposed like this. I think the silent majority of editors who want to see pages load and save quickly/reliably outweighs the need for unlimited use of a very complicated parser functions structure.
Steven
+1. SERIOUSLY. This always comes to mind when these issues come up. Editors and readers (on cache miss) shouldn't have to suffer through this. We shouldn't forget the 99% :)
-- View this message in context: http://wikimedia.7.n6.nabble.com/Some-questions-about-switch-tp4350750p43540... Sent from the Wikipedia Developers mailing list archive at Nabble.com.
Steven Walling wrote:
On Mon, Jan 30, 2012 at 3:23 PM, Tim Starling tstarling@wikimedia.orgwrote:
I'm considering introducing a limit on #switch cases of 2000 or so per article, to address this issue. No doubt many templates will break, but it's important to protect our servers, and we've always discouraged this kind of #switch application.
As a Wikipedian, I would be very happy to see a limitation imposed like this. I think the silent majority of editors who want to see pages load and save quickly/reliably outweighs the need for unlimited use of a very complicated parser functions structure.
This is probably going to read like an attack, but it isn't one.
I continue to view the existence of "expensive parser functions" as a failure. I don't believe that having users focus on such details is useful or necessary and I think it largely distracts from any wiki's primary mission. While these types of functions make a page render/load take longer (and this unquestionably needs to be addressed), it's more than a vocal and active minority who favor templates that can do automatic math conversions and unit labeling, string parsing for automatic page title italics and the like, and more. In some ways, if these templates weren't so popular, there'd be no issue. But they're popular for a reason (even if some are only popular among hardcore template nerds, the result is the same...).
As I recall, it wasn't an overwhelming amount of time that was needed to fix #ifexist to batch its queries rather than imposing a limit on it (it was #ifexist that brought the idea of expensive parser functions into existence, as I remember it... I'm too lazy to hunt down revs). Implementing the latter was a more interesting coding challenge, I think, and there was a thought that the methodology used could lay the framework for other parser functions to be limited going forward (which inevitably happened).
However, I believe this was the wrong approach then and continues to be the wrong approach now. The time spent here developing ways to limit #switch rather than fixing the underlying issue (lack of a proper way to do advanced string manipulation, conversion, etc. in templates) seems like a waste of time to me, particularly with a better way (Lua) now possibly on the actual horizon.
And, for what it's worth, Aaron's comment about the 99% is kind of bullshit. The 99% aren't editing and are (instead) hitting cache layers. They're completely unaffected by the slow rendering time. If the 99% were getting the slow rendering times, this issue would have been solved years ago. It's because it only hits a subset of users (and a subset of articles) that it's lingered for as long as it has.
MZMcBride
By 99%, I meant 99% of users don't care (much or at all) about the things these crazy templates tend to offer, they just want to read the article. I remember Domas complaining about this last hackathon when...fixing...ocwiki.
Any article that uses slow templates that take forever to render is hard to edit (since there is always a fresh parse). Making such pages hard to edit or slow to view for users who are logged in and might have a few custom preferences or otherwise have a generic cache miss is pretty disappointing. It also is discouraging to new editors trying to change the page. Maybe it's a result of the "don't care about performance" policy taken to extremes.
I've been pushing for Lua for months (rather than delay on JS vs Lua), and I'm glad it's gotten steam again. Hopefully it will make these problems moot. I'm also glad to see the increased focus on new editors in general (from the features team).
-- View this message in context: http://wikimedia.7.n6.nabble.com/Some-questions-about-switch-tp4350750p43569... Sent from the Wikipedia Developers mailing list archive at Nabble.com.
Thanks for contributions. Really I was going to use large switches, both as associative arrays and as sets. I hoped, that algorithm was based simply on string search into the code (I presume, it is possible, and I know how efficient is plain string search in any decent language) but I guess, from your replies, that I was wrong. :-(
I'll use strictly the 2000 items limit; I presume that - in occasional cases - I will be forced to split larger switches into different containers, after some string manipulation with padleft trick.
Well, it's hyronical that wiki is based on a powerful database, while user interface forces contributors to use any kind of dirty trick to simulate basic, extremely simple database-like jobs ..... :-(
Alex
I'm considering introducing a limit on #switch cases of 2000 or so per article, to address this issue. No doubt many templates will break, but it's important to protect our servers, and we've always discouraged this kind of #switch application.
In my opinion, we shall first check and list which templates will break, suggesting to fix them as soon as possible. Because of so many cases distincted, a 2000 cases #switch should be a overused template inside its wiki. Then introducing the limit.
Nickanc [[m:User:Nickanc]]
On 31 January 2012 14:05, Nickanc Wikipedia nickanc.wiki@gmail.com wrote:
I'm considering introducing a limit on #switch cases of 2000 or so per article, to address this issue. No doubt many templates will break, but it's important to protect our servers, and we've always discouraged this kind of #switch application.
In my opinion, we shall first check and list which templates will break, suggesting to fix them as soon as possible. Because of so many cases distincted, a 2000 cases #switch should be a overused template inside its wiki. Then introducing the limit.
Nickanc [[m:User:Nickanc]]
Could this be implemented through a parser tracking category? IIRC the preprocessor generates other tracking cats when it hits its existing limits, can it easily add a category *without* affecting the rendering?
--HM
On Tue, Jan 31, 2012 at 10:23:39AM +1100, Tim Starling wrote:
I'm considering introducing a limit on #switch cases of 2000 or so per article, to address this issue. No doubt many templates will break, but it's important to protect our servers, and we've always discouraged this kind of #switch application.
Is the problem too many switch cases on the page, or just too many switch cases in any one switch? Does that proposed limit mean that 100 uses of a template with 20 #switch cases will break, too? Will the limit count {{#switch:{{{1}}}|a|b|c=foo}} as 1 or 3 cases?
And will the obvious on-wiki fix of using nested #ifs just make things worse?
Roan Kattouw wrote:
https://oc.wikipedia.org/w/index.php?title=Mod%C3%A8l:Altmejcom&action=e... has 36,611 cases, because it's essentially used as a database.
Looks like a basic "key => value" store which seems to be a valid usage.
Could that use case be fulfilled by semantic mediawiki? Should we create an extension that would let users create key => value stores?
On Wed, Feb 1, 2012 at 7:02 PM, Antoine Musso hashar+wmf@free.fr wrote:
Looks like a basic "key => value" store which seems to be a valid usage.
Could that use case be fulfilled by semantic mediawiki? Should we create an extension that would let users create key => value stores?
I assume that proper scripting language would be a solution, if not proper, than at least partial.
--vvv
On 1 February 2012 16:02, Antoine Musso hashar+wmf@free.fr wrote:
Should we create an extension that would let users create key => value stores?
We already have that… See e.g. [[w:Category:Political party colour templates]].
-- [[cs:User:Mormegil | Petr Kadlec]]
On 02/01/2012 04:38 PM, Petr Kadlec wrote:
On 1 February 2012 16:02, Antoine Musso hashar+wmf@free.fr wrote:
Should we create an extension that would let users create key => value stores?
We already have that… See e.g. [[w:Category:Political party colour templates]].
Or UI messages in the MediaWiki namespace, or.. the wiki in general ;)
Gabriel
wikitech-l@lists.wikimedia.org