Hello,
I set up XHGui on the Wikimedia Foundation's production cluster:
https://performance.wikimedia.org/xhgui/
XHGui is a web application for browsing and analyzing PHP profiling data collected via the XHProf hierarchical profiler. It's now available on the Wikimedia Foundation's production cluster:
https://performance.wikimedia.org/xhgui/
You can see an example of a profiled request here:
https://performance.wikimedia.org/xhgui/run/view?id=56ca93be7ed6ccf971b5b693
I set things up so all requests which bear the X-Wikimedia-Debug header (see https://wikitech.wikimedia.org/wiki/X-Wikimedia-Debug) are profiled and entered into XHGui's database.
Anyone can use this setup to profile requests and view profiling data, but for the moment POSTing to XHGui (to add function watches, create custom views, etc.) requires WMF staff or NDA LDAP credentials.
Note that cookie, client IP, form data, and all query parameters except 'action', are stripped from requests. Also note that the app server that handles X-Wikimedia-Debug requests is configured to process messages in all log channels, which grossly inflates the runtime of log-processing functions.