Hello everyone,
I am trying to obtain through a web request from an application i'm working on the raw wikitext version of an arbitrary Wikipedia article. However, i prefer the templates to be already expanded into their constituent wikitext and/or HTML, and from what i read on the Wikimedia API documentation this should be done by appending "&templates=expand" to the query string; thus, if I wanted the English version of the article about Barcelona in wikitext with template expansion, the URL would be: http://en.wikipedia.org/w/index.php?action=raw&title=Barcelona&templ.... However, i soon found out that not all of the templates are actually expanded, the exception being those templates enclosed in wiki tags, which are left unexpanded (perhaps the most common example being {{cite ...}} templates inside <ref></ref> tags).
I eventually learned that the API provides an individual template expansion module, accessible through ...api.php?action=expandtemplates&text={{template_to_expand}} . This was not really the solution I hoped for actually, since it is impractical for long templates (i assume there's a limit imposed on that query string) and for each of the unexpanded template i'd have to send a separate request to the server, which could prove highly expensive in terms of performance. I've also seen there's the special Wikipedia template expansion page (http://en.wikipedia.org/wiki/Special:ExpandTemplates ), but I don't know if and how I could access its functionality from my application and how I could use it for several templates at once.
Could anyone please suggest a solution for expanding all the templates or at least a viable workaround for this issue ? Thanks in advance, Gabriel S.
2010/10/14 Gabriel Sandor gabi.t.sandor@gmail.com:
since it is impractical for long templates (i assume there's a limit imposed on that query string)
You can POST to the API too.
I've also seen there's the special Wikipedia template expansion page (http://en.wikipedia.org/wiki/Special:ExpandTemplates ), but I don't know if and how I could access its functionality from my application and how I could use it for several templates at once.
Special:ExpandTemplates has the same functionality as action=expandtemplates AFAIK.
Roan Kattouw (Catrope)
Thanks for the information. I've tried doing a POST from my .NET application to the API and indeed the templates are expanded and i receive an XML response. There is, however, a problem. If i'm sending several templates to be expanded at once (say, about 100), all appended one after the other, the Wikipedia API does not return the expansion for all of them - the result is still a well-formed XML, but the template expansion stops at some point (i can provide concrete examples if needed). On the other hand, asking for expansion of the same concatenated templates through the Wikipedia's Special:ExpandTemplates page works as expected, all templates are expanded. Since it seems that both approaches are based on the same functionality, what could I be doing wrong in my application ? Are there any "HTTP settings" that need to be done, such as headers ? Or is it the Wikipedia API which restricts the length of the response ?
On Thu, Oct 14, 2010 at 7:54 PM, Roan Kattouw roan.kattouw@gmail.comwrote:
2010/10/14 Gabriel Sandor gabi.t.sandor@gmail.com:
since it is impractical for long templates (i assume there's a limit imposed on that query string)
You can POST to the API too.
I've also seen there's the special Wikipedia template expansion page (http://en.wikipedia.org/wiki/Special:ExpandTemplates ), but I don't
know if
and how I could access its functionality from my application and how I
could
use it for several templates at once.
Special:ExpandTemplates has the same functionality as action=expandtemplates AFAIK.
Roan Kattouw (Catrope)
Mediawiki-api mailing list Mediawiki-api@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-api
On Thu, Oct 14, 2010 at 4:52 PM, Gabriel Sandor gabi.t.sandor@gmail.com wrote:
However, i soon found out that not all of the templates are actually expanded, the exception being those templates enclosed in wiki tags, which are left unexpanded (perhaps the most common example being {{cite ...}} templates inside <ref></ref> tags).
It appears to be that this happens indeed for <ref> tags. These tags are part of extension Cite. Cite normally takes those tags, removes them an relocates them to the location of the <references /> tag.
I'm guessing here, but what I think happens is that extension Cite only does this in the parsing stage, so in the preprocessing stage nothing is processed. Perhaps somebody with parser and Cite knowledge can elaborate a bit more on this?
Bryan
The parser converts tags to __UNIQ__ tokens while parsing and then expands these (either to original input, or to extension output) right at the end. Because you are doing expandtemplates, the __UNIQ__ tokens are expanded to the original wikitext instead of the rendered contents.
Tag extension output is arbitrary HTML, so putting the rendered contents would not be correct for just "expandtemplates" in many cases. Adding another mechanism to the tag hooks API so that they can opt to return "this is the wikitext I would have rendered" would be possible, if confusing (we'd then have some tag hooks returning input wikitext if they didn't, or couldn't realistically, support the new API).
Conrad
On 15 October 2010 11:07, Bryan Tong Minh bryan.tongminh@gmail.com wrote:
On Thu, Oct 14, 2010 at 4:52 PM, Gabriel Sandor gabi.t.sandor@gmail.com wrote:
However, i soon found out that not all of the templates are actually expanded, the exception being those templates enclosed in wiki tags,
which
are left unexpanded (perhaps the most common example being {{cite ...}} templates inside <ref></ref> tags).
It appears to be that this happens indeed for <ref> tags. These tags are part of extension Cite. Cite normally takes those tags, removes them an relocates them to the location of the <references /> tag.
I'm guessing here, but what I think happens is that extension Cite only does this in the parsing stage, so in the preprocessing stage nothing is processed. Perhaps somebody with parser and Cite knowledge can elaborate a bit more on this?
Bryan
Mediawiki-api mailing list Mediawiki-api@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-api
Thanks for your answers.... So basically you're saying that for no wikitag the templates that are enclosed by it are guaranteed to be expanded ? Or is it just for <ref> and a few other wikitags ? Or is it rather for specific combinations of wikitags and templates ? I'm sorry for keep asking this but unfortunately i couldn't find anything in the documentation that could make it clear for me. I'm experimenting now with the Special:ExpandTemplates page but i still can't reach a definite conclusion.
On Fri, Oct 15, 2010 at 1:47 PM, Conrad Irwin conrad.irwin@gmail.comwrote:
The parser converts tags to __UNIQ__ tokens while parsing and then expands these (either to original input, or to extension output) right at the end. Because you are doing expandtemplates, the __UNIQ__ tokens are expanded to the original wikitext instead of the rendered contents.
Tag extension output is arbitrary HTML, so putting the rendered contents would not be correct for just "expandtemplates" in many cases. Adding another mechanism to the tag hooks API so that they can opt to return "this is the wikitext I would have rendered" would be possible, if confusing (we'd then have some tag hooks returning input wikitext if they didn't, or couldn't realistically, support the new API).
Conrad
On 15 October 2010 11:07, Bryan Tong Minh bryan.tongminh@gmail.comwrote:
On Thu, Oct 14, 2010 at 4:52 PM, Gabriel Sandor gabi.t.sandor@gmail.com wrote:
However, i soon found out that not all of the templates are actually expanded, the exception being those templates enclosed in wiki tags,
which
are left unexpanded (perhaps the most common example being {{cite ...}} templates inside <ref></ref> tags).
It appears to be that this happens indeed for <ref> tags. These tags are part of extension Cite. Cite normally takes those tags, removes them an relocates them to the location of the <references /> tag.
I'm guessing here, but what I think happens is that extension Cite only does this in the parsing stage, so in the preprocessing stage nothing is processed. Perhaps somebody with parser and Cite knowledge can elaborate a bit more on this?
Bryan
Mediawiki-api mailing list Mediawiki-api@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-api
Mediawiki-api mailing list Mediawiki-api@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-api
Some tags (like <math>) have their own parsers, other's (like <ref>) use the MediaWiki parser with some custom behaviour added. There's no internal mechanism to convert any tag extension into MediaWiki markup that would render the output of the tag extension (there could be for some tags, but there isn't, and probably won't be because there is no markup that works for some tags). Therefore, if you want to parse tags (like <math> or <ref>) you either need to install a MediaWiki or emulate the behaviour manually (for example by parsing the contents of the <ref> section using the API).
Conrad
On 15 October 2010 12:55, Gabriel Sandor gabi.t.sandor@gmail.com wrote:
Thanks for your answers.... So basically you're saying that for no wikitag the templates that are enclosed by it are guaranteed to be expanded ? Or is it just for <ref> and a few other wikitags ? Or is it rather for specific combinations of wikitags and templates ? I'm sorry for keep asking this but unfortunately i couldn't find anything in the documentation that could make it clear for me. I'm experimenting now with the Special:ExpandTemplates page but i still can't reach a definite conclusion.
On Fri, Oct 15, 2010 at 1:47 PM, Conrad Irwin conrad.irwin@gmail.comwrote:
The parser converts tags to __UNIQ__ tokens while parsing and then expands these (either to original input, or to extension output) right at the end. Because you are doing expandtemplates, the __UNIQ__ tokens are expanded to the original wikitext instead of the rendered contents.
Tag extension output is arbitrary HTML, so putting the rendered contents would not be correct for just "expandtemplates" in many cases. Adding another mechanism to the tag hooks API so that they can opt to return "this is the wikitext I would have rendered" would be possible, if confusing (we'd then have some tag hooks returning input wikitext if they didn't, or couldn't realistically, support the new API).
Conrad
On 15 October 2010 11:07, Bryan Tong Minh bryan.tongminh@gmail.comwrote:
On Thu, Oct 14, 2010 at 4:52 PM, Gabriel Sandor gabi.t.sandor@gmail.com wrote:
However, i soon found out that not all of the templates are actually expanded, the exception being those templates enclosed in wiki tags,
which
are left unexpanded (perhaps the most common example being {{cite ...}} templates inside <ref></ref> tags).
It appears to be that this happens indeed for <ref> tags. These tags are part of extension Cite. Cite normally takes those tags, removes them an relocates them to the location of the <references /> tag.
I'm guessing here, but what I think happens is that extension Cite only does this in the parsing stage, so in the preprocessing stage nothing is processed. Perhaps somebody with parser and Cite knowledge can elaborate a bit more on this?
Bryan
Mediawiki-api mailing list Mediawiki-api@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-api
Mediawiki-api mailing list Mediawiki-api@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-api
Mediawiki-api mailing list Mediawiki-api@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-api
mediawiki-api@lists.wikimedia.org