[Wikitech-l] Performance issue with regard to JSON-based localisation format

Hi *, I recently investigated a performance issue with one of Wikibase's test cases. The test in question performed a formatting task for all known languages. While the first assertions ran as fast as expected, they rapidly got slower until finally taking seconds per assertion. I figured this was an actual performance bug (although hardly triggered in production) and started profiling. My findings, in short: * The shims for $wgExtensionMessagesFiles as generated by maintenance/generateJsonI18n.php register a handler for the hook LocalisationCacheRecache when included * For every new language loaded, all $wgExtensionMessagesFiles are included by LocalisationCache::recache * Afterwards, LocalisationCache::recache runs the hook LocalisationCacheRecache This leads to the obvious issue that there is a growing number of registered handlers, which slows down the hook, which slows down the test. implementation of LocalisationCache assume that $wgExtensionMessagesFiles can be included multiple times. However, the $wgExtensionMessagesFiles shims as generated by maintenance/generateJsonI18n.php don't handle this case gracefully. I can imagine two solutions: * Fixing the caller (LocalisationCache) by making sure it does not include a single file twice (a simple, finite cache is already implemented in LocalisationCacheBulkLoad). * Fixing the callees ($wgExtensionMessagesFiles) by making sure they register the handler only once. This would need a change to all extensions which already switched to JSON-based localisation. What do you think? Adrian Lang