Aryeh Gregor wrote:
Ok, I will keep this in mind. Parsing the template on the home wiki seems necessary because it can use other templates hosted on that wiki to render correctly... I think it is the most logical way to do, isn't it?
I think parsing the template on the local wiki is better, because it gives you more flexibility. For instance, it can use local {{SITENAME}} and so forth. {{CONTENTLANG}} would be especially useful, if we're assuming that templates will be transcluded to many languages.
There are imho fewer variables set by the caller wiki, which could be passed with the query.
This doesn't mean that it has to use the local wiki's templates. There would be two ways to approach this:
- Just don't use the local wiki's templates. Any template calls from
the foreign wiki's template should go to the foreign wiki, not the local wiki. If this is being done over the API, then as an optimization, you could have the foreign wiki send back all templates that will be required, not just the actual template requested.
- Use the local wiki's templates, and assume that the template on the
foreign wiki is designed to be used remotely and will only call local templates when it's really desired. This gives even more flexibility if the foreign template is designed for this use, but it makes it harder to use templates that aren't designed for foreign use.
At first glance, it seems to me that (1) is the best -- do all parsing on the local wiki, but use templates from the foreign wiki. This will cause errors if the local wiki doesn't have necessary extensions installed, like ParserFunctions, but it gives more flexibility overall.
Using 1 you could still allow calling the local template by using {{msg:xyz}}
Another issue here is performance. Parsing is one of the most expensive operations MediaWiki does. Nobody's going to care much if foreign sites request a bunch of templates that can be served out of Squid, but if there are lots of foreign sites that are requesting giant infoboxes and those have to be parsed by Wikimedia servers, Domas is going to come along with an axe pretty soon and everyone's sites will break. Better to head that off at the pass.
Probably time to revive the native preprocessor project. We may want to have both ways implemented, with one falling back on the other.
What you're proposing is that Wikimedia servers do this on a cache miss:
- An application server sends an HTTP request to a Squid with
If-Modified-Since.
- The Squid checks its cache, finds it's a miss, and passes the
request to another Squid.
- The other Squid checks its cache, finds it's a miss, and passes the
request to a second application server.
- The second application server loads up the MediaWiki API and sends
a request to a database server.
The database server returns the result to the second application server.
The second application server returns the results to the Squids,
which cache it and return it to the first application server.
- The first application server caches the result in the database.
For intra-Wikimedia query, they could directly ask an apache. They can even send the query to localhost. Using the api seems the completely right approach for remote users, it can be later refined to add more backends.
Anyway, I don't think api request would be cacheable by squids, so it would be directly passed to an application server.
On Tue, May 25, 2010 at 9:22 AM, Platonides Platonides@gmail.com wrote:
He can internally call the api from the other wiki via FauxRequest.
How will that interact with different configuration settings? I thought FauxRequest only handles requests to the current wiki.
Mmh, right. And we are too based on globals to have two mediawiki instances running in the same php environment :(
I'm afraid that it will produce the opposite. A third party downloads a xml dump for offline use but it doesn't work because it needs a dozen templates from meta (in the worst case, templates from a dozen other wikis).
My point is that ideally, you'd be able to copy-paste enwiki pages and then get the templates to work by configuring them to be fetched from enwiki. Even more ideally, you might want to fetch the enwiki templates as of the point in time your page was downloaded, in case the templates changed syntax (and also to allow indefinite caching).
They would need to prepend the interwiki to all template incantations. That sounds like an option to import templates from foreign wiki first time it is used, being automatically updated as long it's not modified locally (skipping the need of the interwiki on the template, but then it'd conflict with local templates).
But I guess that's much better handled by just using a proper export, and having the templates included in that, so never mind.
Yes. Perhaps they could have a Special:ImportFromRemote to do one-click imports.
On Tue, May 25, 2010 at 9:30 AM, Platonides wrote:
Infinite loops could only happen if both wikis can fetch from the other one. A simple solution would be to pass with the query who requested it originally. If the home wiki calls a different wiki, it would blame the one who asked for it (or maybe building a wiki + template path).
An even simpler solution would be to only set up one wiki to allow this kind of foreign template request, the way Commons is set up now. But that might be limiting.
That's how I'd deploy it. But the code should be robust enough to handle the infinite loops that Peter presents.