I'm writing to draw your attention to a newly-available resource for
performance analysis: flame graphs of MediaWiki code.
http://performance.wikimedia.org/xenon/svgs/
Flame graphs are a visualization of application stack traces.
Each application server on the Wikimedia cluster has a software timer that
interrupts MediaWiki once every ten minutes to capture a stack trace. The
stack trace shows what the code the application server was in the process
of executing when the timer expired. A central log aggregator collects
these traces and uses them to generate flame graphs.
Each box in a flame graph represents a function in the stack. The y-axis
shows stack depth. The topmost box shows the function that was on-CPU at
the moment the trace was captured. The function below a function is its
parent.
The x-axis spans the sample population. The width of the box shows the
total time it was on-CPU or part of an ancestry that was on-CPU, based on
sample count.
Here is an example:
http://performance.wikimedia.org/xenon/svgs/hourly/2014-12-15_22.svgz
To learn more about flame graphs and how to interpret them, see <
http://www.brendangregg.com/flamegraphs.html>gt;.