Hi all!
tl;dr: How to best handle the situation of an old parser cache entry not containing all the info expected by a newly deployed version of code?
We are currently working to improve our usage of the parser cache for Wikibase/Wikidata. E.g., We are attaching additional information related to languagelinks the to ParserOutput, so we can use it in the skin when generating the sidebar.
However, when we change what gets stored in the parser cache, we still need to deal with old cache entries that do not yet have the desired information attached. Here's a few options we have if the expected info isn't in the cached ParserOutput:
1) ...then generate it on the fly. On every page view, until the parser cache is purged. This seems bad especially if generating the required info means hitting the database.
2) ...then invalidate the parser cache for this page, and then a) just live with this request missing a bit of output, or b) generate on the fly c) trigger a self-redirect.
3) ...then generated it, attach it to the ParserOutput, and push the updated ParserOutput object back into the cache. This seems nice, but I'm not sure how to do that.
4) ...then force a full re-rendering and re-caching of the page, then continue. I'm not sure how to do this cleanly.
So, the simplest solution seems to be 2, but it means that we invalidate the parser cache of *every* page on the wiki potentially (though we will not hit the long tail of rarely viewed pages immediately). It effectively means that any such change requires all pages to be re-rendered eventually. Is that acceptable?
Solution 3 seems nice and surgical, just injecting the new info into the cached object. Is there a nice and clean way to *update* a parser cache entry like that, without re-generating it in full? Do you see any issues with this approach? Is it worth the trouble?
Any input would be great!
Thanks, daniel
On Tue, Sep 9, 2014 at 12:03 PM, Daniel Kinzler daniel@brightbyte.de wrote:
Hi all!
tl;dr: How to best handle the situation of an old parser cache entry not containing all the info expected by a newly deployed version of code?
We are currently working to improve our usage of the parser cache for Wikibase/Wikidata. E.g., We are attaching additional information related to languagelinks the to ParserOutput, so we can use it in the skin when generating the sidebar.
However, when we change what gets stored in the parser cache, we still need to deal with old cache entries that do not yet have the desired information attached. Here's a few options we have if the expected info isn't in the cached ParserOutput:
- ...then generate it on the fly. On every page view, until the parser
cache is purged. This seems bad especially if generating the required info means hitting the database.
- ...then invalidate the parser cache for this page, and then a) just
live with this request missing a bit of output, or b) generate on the fly c) trigger a self-redirect.
- ...then generated it, attach it to the ParserOutput, and push the
updated ParserOutput object back into the cache. This seems nice, but I'm not sure how to do that.
https://gerrit.wikimedia.org/r/#/c/158879/ is my attempt to update ParserOutput cache entry, though it seems too simplistic a solution.
Any feedback on this would be great or suggestions on how to do this better, or maybe it's crazy idea. :P
Cheers, Katie
- ...then force a full re-rendering and re-caching of the page, then
continue. I'm not sure how to do this cleanly.
So, the simplest solution seems to be 2, but it means that we invalidate the parser cache of *every* page on the wiki potentially (though we will not hit the long tail of rarely viewed pages immediately). It effectively means that any such change requires all pages to be re-rendered eventually. Is that acceptable?
Solution 3 seems nice and surgical, just injecting the new info into the cached object. Is there a nice and clean way to *update* a parser cache entry like that, without re-generating it in full? Do you see any issues with this approach? Is it worth the trouble?
Any input would be great!
Thanks, daniel
-- Daniel Kinzler Senior Software Developer
Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Also option 5 could be to continue without the days until the parser cash is invalidated on its own. Maybe option 6 could be to continue without the data and invalidate the cache and completely rerender only some of the time. Like 5% of the time for the first couple hours then 25% of the time for a day then 100% of the time after that. It'd guarantee that the cache is good after a certain amount of time without causing a big spike ridge after deploys. All those options are less good then just updating the cache I think.
Nik On Sep 9, 2014 6:42 AM, "aude" aude.wiki@gmail.com wrote:
On Tue, Sep 9, 2014 at 12:03 PM, Daniel Kinzler daniel@brightbyte.de wrote:
Hi all!
tl;dr: How to best handle the situation of an old parser cache entry not containing all the info expected by a newly deployed version of code?
We are currently working to improve our usage of the parser cache for Wikibase/Wikidata. E.g., We are attaching additional information related
to
languagelinks the to ParserOutput, so we can use it in the skin when generating the sidebar.
However, when we change what gets stored in the parser cache, we still need to deal with old cache entries that do not yet have the desired information attached. Here's a few options we have if the expected info isn't in the cached ParserOutput:
- ...then generate it on the fly. On every page view, until the parser
cache is purged. This seems bad especially if generating the required info means hitting the database.
- ...then invalidate the parser cache for this page, and then a) just
live with this request missing a bit of output, or b) generate on the fly c)
trigger
a self-redirect.
- ...then generated it, attach it to the ParserOutput, and push the
updated ParserOutput object back into the cache. This seems nice, but I'm not
sure
how to do that.
https://gerrit.wikimedia.org/r/#/c/158879/ is my attempt to update ParserOutput cache entry, though it seems too simplistic a solution.
Any feedback on this would be great or suggestions on how to do this better, or maybe it's crazy idea. :P
Cheers, Katie
- ...then force a full re-rendering and re-caching of the page, then
continue. I'm not sure how to do this cleanly.
So, the simplest solution seems to be 2, but it means that we invalidate the parser cache of *every* page on the wiki potentially (though we will not hit the long tail of rarely viewed pages immediately). It effectively means that any such change requires all pages to be re-rendered eventually. Is that acceptable?
Solution 3 seems nice and surgical, just injecting the new info into the cached object. Is there a nice and clean way to *update* a parser cache entry
like
that, without re-generating it in full? Do you see any issues with this approach? Is it worth the trouble?
Any input would be great!
Thanks, daniel
-- Daniel Kinzler Senior Software Developer
Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
-- @wikimediadc / @wikidata _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikidata-tech@lists.wikimedia.org