Hi all,
I am working on a custom ResourceLoader module whose style contents are compiled from Sass stylesheets. These Sass stylesheets take as input user-defined theme variables that affect the output. I am looking for a way to invalidate ResourceLoader URLs that include such style modules, when these variables change as a result of an user action—primarily so that the user can see their own changes.
ResourceLoader will cache such style URLs for 5 minutes on the CDN and in the user’s browser cache, so by default the user would have to wait until their cached copy of styles expire before they would receive the updated version. Accordingly, I have tried to explore if following the document steps to empty one’s browser cache[1] after making changes would cause the browser to fetch the updated resource from the backend. Unfortunately, both Chrome and Firefox appear to send a “Cache-Control: no-cache” request header for the style URL if the page is hard refreshed, which seems to cause most CDNs (including WMF’s) to act as the source of truth and serve the asset from cache if it is already there, without triggering a backend fetch or conditional revalidation. In the case of a simple refresh, Chromium and derivatives will not trigger a conditional revalidation of resources including on a page since 2017,[2] so they just continue to use the asset that was already in the browser’s cache. Thus, the standard instructions for emptying one’s cache do not seem to be sufficient to allow the user to see the effect of their styling changes.
Given the above, the only option I see that would allow the user to see the updated styles after modifying input variables would be to append a version hash to the style URL included in the page HTML—something which ResourceLoader does not currently do.
Do you know of a better approach to this problem? Are there any examples in MediaWiki core or extensions that accomplish something similar? I have consulted the code for ResourceLoaderSiteStylesModule, but it does not seem to have any specific invalidation behaviour, nor does it use a version hash in style URLs in the page HTML, so it seems that any updates to it take 5 minutes to propagate.
Thanks in advance!
— [1] https://en.wikipedia.org/wiki/Wikipedia:Bypass_your_cache [2] https://bugs.chromium.org/p/chromium/issues/detail?id=505048
Cheers,
Máté Szabó SOFTWARE ENGINEER +36 30 947 5903
Fandom Poland sp. z o.o. z siedzibą w Poznaniu, ul. Abp. A. Baraniaka 6 Sąd Rejonowy Poznań – Nowe Miasto i Wilda w Poznaniu, VIII Wydział Gospodarczy Krajowego Rejestru Sądowego, KRS 0000254365 NIP: 5252358778 Kapitał zakładowy: 50.000,00 złotych
Yes, it takes 5 minutes for just about everything to update in ResourceLoader. This is also true for ResourceLoaderSiteStylesModule: when you save an edit to e.g. MediaWiki:Common.css, you don't immediately see the styles change. However, previewing edits to MediaWiki:Common.css does work, because of some special handling.
When previewing, EditPage sets a content override (using OutputPage::addContentOverride()) for the page being previewed, with the text that's in the edit box. This content override is then propagated to the ResourceLoaderContext object by OutputPage::getRlClientContext(). ResourceLoaderWikiModule checks these content overrides before accessing wiki pages (which is how it gets the content from the edit box instead of the content that's on the wiki page), and it also causes shouldEmbedModule() to return true if there's a content override that applies to it (which is how it avoids requesting the module from load.php and instead embeds it as an inline <script>/<style> tag). The TemplateSandbox extension also uses this, see https://gerrit.wikimedia.org/r/c/mediawiki/extensions/TemplateSandbox/+/3407... .
You may be able to do something similar to allow users to preview changes to the theme variables: use content overrides (or some other mechanism) to get the previewed values to your module, then return true from shouldEmbedModule() when you detect that a preview is happening.
On Wed, Feb 12, 2020 at 10:32 AM Máté Szabó mszabo@wikia-inc.com wrote:
Hi all,
I am working on a custom ResourceLoader module whose style contents are compiled from Sass stylesheets. These Sass stylesheets take as input user-defined theme variables that affect the output. I am looking for a way to invalidate ResourceLoader URLs that include such style modules, when these variables change as a result of an user action—primarily so that the user can see their own changes.
ResourceLoader will cache such style URLs for 5 minutes on the CDN and in the user’s browser cache, so by default the user would have to wait until their cached copy of styles expire before they would receive the updated version. Accordingly, I have tried to explore if following the document steps to empty one’s browser cache[1] after making changes would cause the browser to fetch the updated resource from the backend. Unfortunately, both Chrome and Firefox appear to send a “Cache-Control: no-cache” request header for the style URL if the page is hard refreshed, which seems to cause most CDNs (including WMF’s) to act as the source of truth and serve the asset from cache if it is already there, without triggering a backend fetch or conditional revalidation. In the case of a simple refresh, Chromium and derivatives will not trigger a conditional revalidation of resources including on a page since 2017,[2] so they just continue to use the asset that was already in the browser’s cache. Thus, the standard instructions for emptying one’s cache do not seem to be sufficient to allow the user to see the effect of their styling changes.
Given the above, the only option I see that would allow the user to see the updated styles after modifying input variables would be to append a version hash to the style URL included in the page HTML—something which ResourceLoader does not currently do.
Do you know of a better approach to this problem? Are there any examples in MediaWiki core or extensions that accomplish something similar? I have consulted the code for ResourceLoaderSiteStylesModule, but it does not seem to have any specific invalidation behaviour, nor does it use a version hash in style URLs in the page HTML, so it seems that any updates to it take 5 minutes to propagate.
Thanks in advance!
— [1] https://en.wikipedia.org/wiki/Wikipedia:Bypass_your_cache [2] https://bugs.chromium.org/p/chromium/issues/detail?id=505048
Cheers,
Máté Szabó SOFTWARE ENGINEER +36 30 947 5903
Fandom Poland sp. z o.o. z siedzibą w Poznaniu, ul. Abp. A. Baraniaka 6 Sąd Rejonowy Poznań – Nowe Miasto i Wilda w Poznaniu, VIII Wydział Gospodarczy Krajowego Rejestru Sądowego, KRS 0000254365 NIP: 5252358778 Kapitał zakładowy: 50.000,00 złotych
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hey Roan,
Thank you for your response!
I did stumble upon ContentOverrideCallback. Unfortunately, it doesn’t seem to be an exact fit for this use case. I think I will opt for manual URL purges to ensure timely updates in this scenario
Cheers,
Máté Szabó SOFTWARE ENGINEER
Fandom Poland sp. z o.o. z siedzibą w Poznaniu, ul. Abp. A. Baraniaka 6 Sąd Rejonowy Poznań – Nowe Miasto i Wilda w Poznaniu, VIII Wydział Gospodarczy Krajowego Rejestru Sądowego, KRS 0000254365 NIP: 5252358778 Kapitał zakładowy: 50.000,00 złotych
If your cache layer is APC, and your cached file is PHP (e.g. load.php?querystring), then you can configure APC to always check for a revised file in the backend (opcache.validate_timestamps=1 in php.ini) -- this may solve the issue if it wasn't set previously. Also, you can write your own minimal php script that calls opcache_invalidate($file, true). [1]
Of course this won't help if you're not using the PHP Opcache [2]
[1] https://tideways.com/profiler/blog/fine-tune-your-opcache-configuration-to-a... [2] https://www.php.net/manual/en/book.opcache.php
Greg Rundlett https://eQuality-Tech.com https://freephile.org
On Thu, Feb 13, 2020 at 8:27 AM Máté Szabó mszabo@wikia-inc.com wrote:
Hey Roan,
Thank you for your response!
I did stumble upon ContentOverrideCallback. Unfortunately, it doesn’t seem to be an exact fit for this use case. I think I will opt for manual URL purges to ensure timely updates in this scenario
Cheers,
Máté Szabó SOFTWARE ENGINEER
Fandom Poland sp. z o.o. z siedzibą w Poznaniu, ul. Abp. A. Baraniaka 6 Sąd Rejonowy Poznań – Nowe Miasto i Wilda w Poznaniu, VIII Wydział Gospodarczy Krajowego Rejestru Sądowego, KRS 0000254365 NIP: 5252358778 Kapitał zakładowy: 50.000,00 złotych
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org