Hi all,

TL;DR: When using X-Wikimedia-Debug to profile web requests on Wikimedia wikis, the generated profile information will now include details from "w/index.php", and MWMultiVersion, and things like wmf-config/CommonSettings.php. Details at https://phabricator.wikimedia.org/T180183.

-

The debug profiler provided on Wikimedia production wikis[1] previously could not cover the code that executes before MediaWiki core instantiates ProfilerXhprof, which was in charge of calling `xhprof_enable`. This normally happens within core's Setup.php.

While that point in Setup.php is before any important MediaWiki core logic, it misses out on two other chunks of code:

1. Initialisation of MediaWiki core – This includes entry point code (eg. index.php, PHPVersionCheck), but also the first steps of Setup before Profiler. Such as AutoLoader, vendor, and LocalSettings.php. At WMF, LocalSettings.php loads wmf-config/InitialiseSettings.php and wmf-config/CommonSettings.php.

2. Wrapping of MediaWiki entrypoint – At Wikimedia, the index.php entrypoint is itself further wrapped in something called "multiversion". Multiversion is what determines the wiki ID (eg. "enwiki") and MediaWiki branch (eg. "1.31.0-wmf.25") associated with the current domain (eg. " en.wikipedia.org").

Over the past weeks, I've been refactoring MediaWiki core, wmf-config and Wikimedia's HHVM settings to make we can instrument the above code as part of our performance profiles.

This change happened in three phases:

## 1. Update wmf-config/StartProfiler to.. actually start the profiler!

The file name is somewhat deceptive because traditionally this is (and can) only be used to *configure* the profiler, by assigning $wgProfiler. It makes sense that we cannot instantiate the Profiler subclass from this file, because the classes and run-time configuration are not and cannot be available this early.

However, we don't the Profiler class to record data. The Profiler classes typically obtain their data from native PHP. The one used at WMF is XHProf. Previously, we would assign $wgProfiler['class'] = 'ProfilerXhprof', and then later MediaWiki core instantiates ProfilerXhprof, which then calls xhprof_enable. We now xhprof_enable directly from StartProfiler.php.

This change enabled coverage of code in Setup.php between 'include StartProfiler' and 'Profiler::instance()'. – Mainly: vendor, LocalSettings, wmf-config.

## 2. Update MediaWiki core to include StartProfiler earlier.

It is now the first thing included by Setup.php.

This change enabled coverage of code in Setup.php that previously was before 'include StartProfiler'. – Namely: AutoLoader.php, Defines.php.

## 3. Configure WMF's PHP engine to use auto_prepend_file

This is the big one, and requires a PHP ini setting change. Third parties can follow the same pattern in order to get the same benefits: * Put `xhprof_enable( $flags )`, along with any sampling/conditional logic, in a separate file. * Use it from two places: ** In StartProfiler.php, include using require_once. ** In php.ini, set auto_prepend_file=path/to/profiler.php.

This change enabled coverage of all remaining code. – Namely: multiversion, w/index.php and things like PHPVersionCheck.

## Example

Using cURL:

$ curl -H 'X-Wikimedia-Debug: 1' 'https://en.wikipedia.beta. wmflabs.org/w/load.php?debug=false&modules=startup&only= scripts&forceprofile=1'

Output now includes: - main() # resembles the wrapper at operations/mediawiki-config.git:/w/load.php -- run_init::/srv/mediawiki/multiversion/MWMultiVersion.php -- MWMultiVersion::getMediaWiki -- run_init:/srv/mediawiki/php-../load.php -- run_init::/srv/mediawiki/php-../includes/Setup.php -- run_init::/srv/mediawiki/php-../LocalSettings.php -- run_init::/srv/mediawiki/wmf-config/CommonSettings.php