In a parserfunctions statement like this:
{{#if: {{#some-condition:}} | {{#some-big-computation:}} }}
if #some-condition is false, which of the following should happen?
1. #some-big-computation gets fully evaluated, but results are not displayed 2. #some-big-computation is not evaluated at all
Looks like choice #1 is happening in MW 1.12.0, which is bad if #some-big-computation has side-effects or is very expensive. Is this correct behavior?
Thanks, DanB
Daniel Barrett skrev:
In a parserfunctions statement like this:
{{#if: {{#some-condition:}} | {{#some-big-computation:}} }}
if #some-condition is false, which of the following should happen?
- #some-big-computation gets fully evaluated, but results are not
displayed 2. #some-big-computation is not evaluated at all
Looks like choice #1 is happening in MW 1.12.0, which is bad if #some-big-computation has side-effects or is very expensive. Is this correct behavior?
It's absolutely crucial to short circuit boolen logic at first false result - in cases logically equivalent to AND operations.
Your example contains just such a case, the is, your 1:st expression should behave equivalent to
{{ if {{#some-condition1}} and <--- break here if "false"! {{#some-condition2}} and {{#some-conditionN}} then // do this (else nothing) }}
In for example Delphi the behaviour for boolean expressions is optional. Default setting for "complete boolen evaluation" is = False, that is, the compiler will produce logic which short circuits at first false result. And this short circuit is very powerful for optimizing code execution.
Regards,
// Rolf Lampa
So Rolf, are you saying that the #if statement is broken, or that my #if statement should be changed somehow?
{{#if: {{#some-condition:}} | {{#some-big-computation:}} }}
(Incidentally, my #if statement contains only one boolean expression. {{#some-big-computation:}} is the "THEN" block.)
DanB
-----Original Message----- It's absolutely crucial to short circuit boolen logic at first false result - in cases logically equivalent to AND operations.
Daniel Barrett skrev:
So Rolf, are you saying that the #if statement is broken, or that my #if statement should be changed somehow?
{{#if: {{#some-condition:}} | {{#some-big-computation:}} }}
(Incidentally, my #if statement contains only one boolean expression. {{#some-big-computation:}} is the "THEN" block.)
DanB
-----Original Message----- It's absolutely crucial to short circuit boolen logic at first false result - in cases logically equivalent to AND operations.
Hm. As a matter of fact, it's a difference between should and is. The #if statement SHOULD be regarded as broken, BUT, ParserFunctions seems to have been struggling with this problem for a long time, and for this reason it's not a technical "bug".
However, ParserFunctions really should be fixed to produce short circuit behaviour. Such a change will not change the logical end result in any way, it will only optimize existing code. I think this (potential for optimization) has been discussed earlier but seemingly none has fixed it as yet.
Since I have built a ParserFunction evaluator in Delphi Pascal in order to mimix MWs ditto I know that the optimization potential is very big. The parser now expands so many templates not used in the end result because of it's "complete boolen evauation" behaviour.
So again: The #if statement is not really "broken" (since it behaves as currently designed), BUT your #if statement really SHOULD be changed. =)
Regards,
// Rolf Lampa
Thanks for your analysis, Rolf.
I just looked at the PHP source of ParserFunctions, however, and #if is not evaluating anything:
function ifHook( &$parser, $test = '', $then = '', $else = '' ) { if ( $test !== '' ) { return $then; } else { return $else; } } $wgParser->setFunctionHook( 'if', array( &$wgExtParserFunctions, 'ifHook' ) );
So the values of $then and $else are getting evaluated somewhere else. Does this mean the issue lies with MediaWiki, not with ParserFunctions?
DanB
Daniel Barrett wrote:
Thanks for your analysis, Rolf.
I just looked at the PHP source of ParserFunctions, however, and #if is not evaluating anything:
function ifHook( &$parser, $test = '', $then = '', $else = '' ) { if ( $test !== '' ) { return $then; } else { return $else; } } $wgParser->setFunctionHook( 'if', array( &$wgExtParserFunctions, 'ifHook' ) );
So the values of $then and $else are getting evaluated somewhere else. Does this mean the issue lies with MediaWiki, not with ParserFunctions?
Now you are dragging me out on deep waters (I'm not a PHP coder really), since I actually didn't analyze the code itself, only the end result of parser behavior (when I mimicked the MW parser in Delphi Pascal code, including (most of) the ParserFunctions).
But even if I didn't analyze the PHP code very much (I'm a Delphi programmer) I still have some thoughts on where to expect to find a clue for the current behavior:
TEMPLATE ENGINE RULES
I do suspect that an answer to why *all* expressions are evaluated (unconditionally) can be found in the history of MW, which must have been to make a "template engine". That concept comes with some philosophic implications as to what a template engine should handle (which would also affect HOW it executes its internal logic).
The concept of template engine leads me to expect to find that MW parser first would collect (strip out) all expressions from the text (Wiki syntax, MAGICWORDs, HTML, ParserFunctions etc) and then manage a list of all the expressions found, which it eventually simply evaluates disregarding any logic in them (a "pure" template engine would never regard active logic anyway).
So such straight-forward execution is probably what to *expect* from a template engine, since "active programmatic" constructs SHOULD not be part of a template engine (well, this is a view in one school of thought on template engines).
However, now the reality is, despite the "ideal case" for a template engine, that there IS "active code" (ParserFunctions) involved in MW, and thus it's important how to actually evaluate boolean expressions. Expanding templates is now sometimes *very* costly.
So, with this "ideal template engine" background in mind I think that *possibly* the original design decision to NOT evaluate(parse) ParserFunction statements "conditionally" stems from the fact that the original template engine really should not have any such "active logic". But remark, this is only my guess. =)
In any case, MW Parser is no longer only a "pure" template engine, but good or bad, I don't see any possible way back to "pure" a template engine. At least not without immense amounts of manual work to refactor existing templates.
Regards,
// Rolf Lampa
Daniel,
It sounds to me like Rolf is saying that your big calculation IS getting executed. That the way the parser function works now it doesn't kick out when your conditional returns false; it processes the then and just doesn't display it.
-Courtney
Christensen, Courtney skrev:
Daniel,
It sounds to me like Rolf is saying that your big calculation IS getting executed. That the way the parser function works now it doesn't kick out when your conditional returns false; it processes the then and just doesn't display it.
Correct (unfortunately). =)
Regards,
// Rolf Lampa
Rolf Lampa skrev:
However, ParserFunctions really should be fixed to produce short circuit behaviour. Such a change will not change the logical end result in any way, it will only optimize existing code. I think this (potential for optimization) has been discussed earlier but seemingly none has fixed it as yet.
Brion Vibber just confirmed that this HAS been fixed, from version 1.12 (see Vibbers post)
This is really good news!
// Rolf Lampa
Rolf Lampa wrote:
Rolf Lampa skrev:
However, ParserFunctions really should be fixed to produce short circuit behaviour. Such a change will not change the logical end result in any way, it will only optimize existing code. I think this (potential for optimization) has been discussed earlier but seemingly none has fixed it as yet.
Brion Vibber just confirmed that this HAS been fixed, from version 1.12 (see Vibbers post)
This is really good news!
// Rolf Lampa
These good news were announced four mounths ago by Tim Starling in the "New Preprocessor" thread :)
http://permalink.gmane.org/gmane.science.linguistics.wikipedia.technical/354...
Rolf Lampa wrote:
this short circuit is very powerful for optimizing code execution.
It's also essential to allow common programming approaches such as:
if( A exists and A isblort() ) {...}
Complete boolean evaluation would fail in this case if A doesn't exist and would force two if statements. So many languages permit short-circuit evaluation that I'm surprised to find out that Delphi allows the opposite as an option.
I would never have expected Parserfunctions to not short-circuit.
Mike
Michael Daly skrev:
Rolf Lampa wrote:
this short circuit is very powerful for optimizing code execution.
It's also essential to allow common programming approaches such as:
if( A exists and A isblort() ) {...}
Yes. But is operator absent expressions (user defined operators like "A isblort") possible in PHP? (It's not supported in Delphi, that's why I didn't come to think of it).
Even short circuit behavior should allow user defined operators (I think it's supported in C#), given that the interpreter can handle , it only doesn't care
Complete boolean evaluation would fail in this case if A doesn't exist and would force two if statements.
Oh, this is the weaknesses with non "Strong typed" languages. However, why not interpret undefined expressions as "false"?
So many languages permit short-circuit evaluation that I'm surprised to find out that Delphi allows the opposite as an option.
Possibly for backward compatibility reasons (I don't know for sure though). I would never use "complete boolean evaluation" myself though, never.
I would never have expected Parserfunctions to not short-circuit.
It's very frustrating although I think there's a reason for this in the first place (MW actually being a "template engine", in which you would NOT expect to evaluate "active logic"). So I guess we shouldn't be too surprised, only frustrated... =)
Regards,
// Rolf Lampa
Michael Daly skrev:
Rolf Lampa wrote:
But is operator absent expressions (user defined operators like "A isblort") possible in PHP?
What I wrote was intended as structured English rather than a specific language.
Ups, since I'm not a native English speaker I'm not sure of what you really mean.
Lets say that "Exist" is a variable or a constant and "IsBlort()" is a function, then the expression below still works as expected in a short-circuited evaluation, like so:
// IsBlort will be evaluated only if Exist = true // otherwise nothing happens.
if ( Exist *and* IsBlort() ) { // code will execute here only if both Exist and IsBlort = true }
Is this what you meant?
Regards,
// Rolf Lampa
Daniel Barrett wrote:
In a parserfunctions statement like this:
{{#if: {{#some-condition:}} | {{#some-big-computation:}} }}
if #some-condition is false, which of the following should happen?
- #some-big-computation gets fully evaluated, but results are not
displayed 2. #some-big-computation is not evaluated at all
#2 (logical shortcut).
I can confirm this is how it works in both 1.12 and trunk (1.13 dev branch).
Looks like choice #1 is happening in MW 1.12.0, which is bad if #some-big-computation has side-effects or is very expensive. Is this correct behavior?
Are you perhaps running with the old parser enabled?
-- brion vibber (brion @ wikimedia.org)
Brion Vibber wrote:
Daniel Barrett wrote:
In a parserfunctions statement like this:
{{#if: {{#some-condition:}} | {{#some-big-computation:}} }}
if #some-condition is false, which of the following should happen?
- #some-big-computation gets fully evaluated, but results are not
displayed 2. #some-big-computation is not evaluated at all
#2 (logical shortcut).
I can confirm this is how it works in both 1.12 and trunk (1.13 dev branch).
Wow! thank you for this news! =)
Regards,
// Rolf Lampa
Hmmm, on our 1.12.0 installation, when we do:
{{#if: {{#some-condition:}} | {{#some-big-computation:}} }}
then #some-big-computation is ALWAYS evaluated (but not necessarily displayed), even if #some-condition is false. I will try to create a test case where #some-big-computation has an obvious side-effect (like creating a file on disk) and show that it happens even when #some-condition is false.
You asked if we might have the old parser enabled. How would this be possible to do? (Or how to confirm we're using the new parser?)
DanB
-----Original Message----- Brion Vibber writes:
- #some-big-computation is not evaluated at all
#2 (logical shortcut).
I can confirm this is how it works in both 1.12 and trunk (1.13 dev branch).
Attached below is a test case that demonstrates the "always evaluate" behavior I was describing in vanilla MediaWiki 1.12.0 and ParserFunctions. The custom parser function #bugtest does two things:
1. Displays a simple message, "Here I Am!" 2. Appends a timestamp to the file C:\Temp\sideeffect.txt (you can change this in the code)
When you call it like this:
{{#if: foo | {{#bugtest:}} }}
it displays "Here I Am!" and writes a timestamp to the file, as you'd expect. However, if you call it like this:
{{#if: | {{#bugtest:}} }}
it does not display any message, but it STILL writes the timestamp, demonstrating that the #bugtest code is being run.
This is on Windows 2003 Server with PHP 5.2.5.
Brion, any comments?
DanB
====
<?php
$vpIgnoreConditional = new IgnoreConditional; $wgExtensionFunctions[] = array($vpIgnoreConditional, 'setup'); $wgHooks['LanguageGetMagic'][] = array($vpIgnoreConditional, 'magic');
class IgnoreConditional { const NAME = 'bugtest'; const OUTFILE = "c:\temp\sideeffect.txt";
// The entry point. Associates the parser function with a PHP function. function setup() { global $wgParser; $wgParser->setFunctionHook(self::NAME, array($this, 'renderParserFunction')); }
function magic(&$magicWords, $langCode) { $magicWords[self::NAME] = array(0, self::NAME); return true; }
// The function that does the work. function renderParserFunction(&$parser) { // Do a side-effect $handle = fopen(self::OUTFILE, "a") or die("Cannot open"); fwrite($handle, date('r') . "\r\n") or die("Cannot write"); fclose($handle);
// Return some text return 'Here I am!'; } }
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Daniel Barrett wrote:
Attached below is a test case that demonstrates the "always evaluate" behavior I was describing in vanilla MediaWiki 1.12.0 and ParserFunctions. The custom parser function #bugtest does two things:
[snip]
it does not display any message, but it STILL writes the timestamp, demonstrating that the #bugtest code is being run.
I can confirm that the timestamp *IS NOT* written for the false case in your example, on trunk, on 1.12 current branch, and 1.12.0 stock release.
Tested on Mac OS X 10.5, PHP 5.2.5.
Possibility 1:
* You switched in the old parser, which does not do this short circuiting (the new parser preprocessor which allows this is the biggest new feature in 1.12):
$wgParserConf['class'] = 'Parser_OldPP';
Possibility 2:
* You're running an old version of ParserFunctions which doesn't know how to plug in to the new parser preprocessor.
The versions in both trunk and branches/REL1_12 seem fine, I tested them both, so I don't know where you might have gotten an old one.
- -- brion vibber (brion @ wikimedia.org)
Thanks so much for trying out my test case. I'm glad that you had different results: it gives me hope for improving things on my end.
The version of ParserFunctions we use is from http://meta.wikimedia.org/wiki/ParserFunctions, which is pointed to by the "table of functions" on http://www.mediawiki.org/wiki/Parserfunctions.
Looking around some more, I now see http://www.mediawiki.org/wiki/Extension:ParserFunctions, which appears to be much newer. I will install that and see if the problem goes away.
Thanks again. DanB
-----Original Message----- Possibility 2:
* You're running an old version of ParserFunctions which doesn't know how to plug in to the new parser preprocessor.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Daniel Barrett wrote:
Thanks so much for trying out my test case. I'm glad that you had different results: it gives me hope for improving things on my end.
The version of ParserFunctions we use is from http://meta.wikimedia.org/wiki/ParserFunctions, which is pointed to by the "table of functions" on http://www.mediawiki.org/wiki/Parserfunctions.
That's just a documentation page, you should end up with the same code.... unless of course you're downloading the old versions it refers to for MediaWiki 1.6 / PHP 4!
- -- brion vibber (brion @ wikimedia.org)
The latest version of ParserFunctions from svn solved everything.
Thanks Brion!!
DanB
Brion Vibber wrote:
Possibility 2:
- You're running an old version of ParserFunctions which doesn't know
how to plug in to the new parser preprocessor.
The versions in both trunk and branches/REL1_12 seem fine, I tested them both, so I don't know where you might have gotten an old one.
ParserFuncions don't came with Mediawiki, something which has lead to fustration on many mw admins here, which take them as forming part of wikitext. As people have to separately download them, some people is probably not updating them when they update mediawiki. Also, there're chances that they're getting an old tar from some site (some distro repository perhaps?).
mediawiki-l@lists.wikimedia.org