Jens Frank wrote: Is this the case in all the Wikipedias? Could this be an explanation of some of the recent performance problems -- some of the templates must be used on thousands, if not tens of thousands, of pages? -- Neil

Timwi wrote: Of course I did. In MediaWiki 1.2 I deliberately avoided this kind of cache purging, for performance reasons. But this behaviour was very unpopular, and some automated method was often requested. So in 1.3 I invalidated the cache of all pages linking to a given template page, when the template page is updated. The purge operation required is no worse than that for page creation or deletion. There are a number of workarounds. Firstly, the cache invalidation only occurs when pages in the Template (Sjabloon) namespace are updated. So you could put the -noun- template in [[WikiWoordenboek:-noun]], and include it with {{WikiWoordenboek:-noun}}. The second alternative is to stop using templates for each and every little link. Are they really necessary? The third alternative is to get it right the first time, to avoid changing it at all costs, and if you really need to change it, only do so in off-peak times. In any case, there isn't a zillion pages using that template, only 497. Perhaps we need to do some database optimisation so that relatively small operations like this become possible. -- Tim Starling

Timwi wrote: "Touch" is the name of a unix utility which changes the modification time on a file, usually setting it to the present time. Hence "touching" a file is equivalent to making some infinitesimal change to it. In the cur table, there is a field called "cur_touched". This field is used for cache invalidation. If links on the page change from broken to unbroken or vice versa, cur_touched is set to the present. This field is queried to determine whether the client cache or the parser cache is valid. The squid cache does not check back with the database on each query, instead it keeps its own records of which pages are valid and which are invalid. So whenever a page changes, we need to not only update cur_touched, but also to send a message to each squid server informing it that the page has become invalid. This is called "purging". The parser cache is a cache of the content HTML of a page, stored in memcached. That is, it omits the sidebar, header and footer. It is stored under an index which indicates what user preferences were used to render it. You can tell when a page is served from the parser cache by looking for an HTML comment at the end of the content area, e.g. <!-- Saved in parser cache with key enwiki:pcache:idhash:13173-1!1!1!monobook!1!1!0!1!0!1!0 and timestamp 20040622004805 --> 13173 is the main page. You can't store anything in memcached unless you're prepared to have it go missing. Memcached is appropriate for caching read operations, but not for replacing write operations. One alternative is to use a system where two cur_touched fields must be checked to validate a page, instead of one. One for the page, one for the template. This would only be efficient for widely included templates. Templates included in only a few pages would have to be handled the old way. Another alternative would be to reduce service. When changing a template, before attempting to invalidate all caches, it would check how many pages are using the template. If it is greater than some critical number, the invalidation could simply be skipped. After a few days, the squid cache and the parser cache will expire, and users will see the updated template. Please don't reply by saying "yeah, do that!" -- Tim Starling