recursiveTagParse makes data vanish - Wikitech-l

16 Nov 2010

Hello. I'm developing a Semantic MediaWiki extension to partition an #ask query
based upon the first character of one of the queried properties so that the user
can page through partitions of what would otherwise be a very large query
result. For example, I'd like to query all pages in Category:Author in my wiki,
but there are almost 4,000 of those, so I want to partition the results
dynamically by the first character of the author's last name. I have a working
version, but I'm in the process of enhancing it to use Ajax so that it only
needs to refresh the <div> with the new query results rather than the entire
page as well as retaining previous partitions in hidden <div>'s so previous
queries don't need to be repeated. I'm having two parsing issues in the Ajax
callback. I do not believe that they are related to Ajax per se. The second
issue may be caused by my inexpert fix to the first issue, although I ran across
the same second issue previously in a different context. I have reproduced both
issues in a vastly simplified, minimal test program that I will discuss below.

I'm using MediaWiki 1.16.0 and SemanticMediaWiki 1.5.2.

The first issue is that I need access to the parser in the Ajax callback so I
can call recursiveTagParse on the results for the query. I tried using
$wgParser, but it does not appear to be in a good state in the callback. I had a
series of errors where the parser was calling functions on non-objects. I played
around with it a bit and was able to get past all of these errors with this
rather ugly code:

    global $wgParser;
    $wgParser->mOptions = new ParserOptions;
    $wgParser->initialiseVariables();
    $wgParser->clearState();
    $wgParser->setTitle(new Title($title));

where $title is the title retrieved using getTitle on the original (pre-Ajax)
working parser and sent as a parameter to JavaScript and back. These calls allow
me to get a parser that does not give me run-time errors, but I'm not at all
confident that this is the correct approach.

The second issue I'm having is that when I call recursiveTagParse on the data
returned from the query in the Ajax callback, some of the data is "disappears".
I should note that I'm invoking the same query and parsing code in both the
initial display of the page and in the Ajax callback. It works initially but
does not work in the callback using the questionable parser instance described
above. However, I had also seen the same blanking behavior in a previous task
where I did not have the first contributing issue. In that case, I changed my
approach completely to avoid the recursiveTagParse, but I don't believe I have
the option of doing that here. I found a bug report that seems similar, but I'm
not sure if it is the same problem, and there doesn't seem to be any movement on
that bug: https://bugzilla.wikimedia.org/show_bug.cgi?id=24556.

I'm reproducing my simplified test case below. You can run it by placing the
following wikitext on a page:

{{#testQuery:[[Category:Author]]
|limit=5
|searchlabel=
|format=table
}}

replacing [[Category:Author]] with the query term of your choice. When you
install the PHP extension code below, you'll see a page with a button and a
table containing up to 5 query results. When you press the button, the same
query is invoked, but the table is empty. If you comment out the line that
contains the call to recursiveTagParse in the PHP code below, you'll see that
the query results are indeed being returned in both cases, but they are getting
obliterated in the Ajax callback by recursiveTagParse.

It may very well be that solving the first issue will solve the second one, but
since I saw the second issue in another context previously, I wonder if they
really are two separate issues.

Thank you very much for any assistance in working through these issues!

Cindy

PHP extension code:

<?php

/**
* To activate the functionality of this extension include the following
* in your LocalSettings.php file:
* include_once("$IP/extensions/TestQuery/TestQuery.php");
*/

if( !defined( 'MEDIAWIKI' ) ) die( "This is an extension to the MediaWiki
  package and cannot be run standalone." );

# credits
$wgExtensionCredits['parserhook'][] = array (
  'name' => 'TestQuery',
  'version' => '1.0',
  'author' => "Cindy Cicalese",
  'description' => "Bug test"
);

$wgUseAjax = true;
$wgAjaxExportList[] = 'testQueryPopulateDiv';

$wgHooks['LanguageGetMagic'][] = 'wfExtensionTestQuery_Magic';
$wgHooks['ParserFirstCallInit'][] = 'efTestQueryParserFunction_Setup';

function efTestQueryParserFunction_Setup (& $parser) {
  $parser->setFunctionHook('testQuery', 'testQuery');
  return true;
}

function wfExtensionTestQuery_Magic(& $magicWords, $langCode) {
  $magicWords['testQuery'] = array (0, 'testQuery');
  return true;
}

function testQuery($parser, $query) {
  $params = func_get_args();
  array_shift($params); // first is $parser; strip it
  array_shift($params); // second is query string; strip it
  $testQuery = new TestQuery();
  $output = $testQuery->firstVisit($parser, $query, $params);
  $parser->disableCache();
  return array($parser->insertStripItem($output, $parser->mStripState),
    'noparse' => false);
}

function testQueryPopulateDiv($query, $paramString, $title) {
  $params = explode("|", $paramString);
  $testQuery = new TestQuery();
  $output = $testQuery->populateDiv($query, $params, $title);
  return $output;
}

class TestQuery {

  private $template = false;

  function firstVisit($parser, $query, $params) {
    $js = <<<EOT
<script type="text/javascript">
function buttonClicked() {
  var query = document.forms['TestQuery'].Query.value;
  var params = document.forms['TestQuery'].Params.value;
  var title = document.forms['TestQuery'].Title.value;
  var div = document.getElementById('TestDiv');
  sajax_do_call('testQueryPopulateDiv', [query, params, title], div);
}
</script>
EOT;
    $parser->mOutput->addHeadItem($js);
    $this->parseParameters($params);
    $output = $this->buildForm($query, $params, $parser->getTitle());
    $result = $this->getData($parser, $query, $params);
    $output .= $this->buildDiv($result);
    return $output;
  }

  function populateDiv($query, $params, $title) {
    global $wgParser;
    $wgParser->mOptions = new ParserOptions;
    $wgParser->initialiseVariables();
    $wgParser->clearState();
    $wgParser->setTitle(new Title($title));
    $this->parseParameters($params);
    $currentPartitionData = $this->getData($wgParser, $query, $params);
    return $currentPartitionData;
  }

  private function parseParameters($params) {
    foreach ($params as $param) {
      if (preg_match("/^ *format *= *template *$/", $param) === 1) {
        $this->template = true;
      }
    }
  }

  private function buildForm($query, $params, $title) {
    $paramString = implode("|", $params);
    $out = <<<EOT
<center>
<button type='button' id='TestButton'
onClick="buttonClicked()">Test Button
</button>
<form id='TestQuery' method='post' action=''>
<input type='hidden' name='Query' value='$query'>
<input type='hidden' name='Params' value='$paramString'>
<input type='hidden' name='Title' value='$title'>
</form>
</center><br>
EOT;
    return $out;
  }

  private function getData($parser, $query, $params) {
    $result = $this->doSMWAsk($parser, $query, $params);
    $result = $parser->recursiveTagParse($result);
    return $result;
  }

  private function doSMWAsk($parser, $query, $rawParams) {
    SMWQueryProcessor::processFunctionParams($rawParams, $qs, $params,
      $printouts);
    $output = SMWQueryProcessor::getResultFromQueryString($query, $params,
      $printouts, SMW_OUTPUT_WIKI);
    return $output;
  }

  private function buildDiv($result) {
    $output = "<div id='TestDiv' display='block'>";
    $output .= $result;
    $output .= "</div>";
    return $output;
  }
}

--
Dr. Cynthia Cicalese
Lead Software Systems Engineer
The MITRE Corporation