I'm starting a new RFC to discuss ways we can improve our PHP profiling.
https://www.mediawiki.org/wiki/Requests_for_comment/Better_PHP_profiling
Please feel free to help expand and/or comment on the talk page if you've got ideas :)
-Chad
I have also been thinking about the profiling system recently, xhprof is a really nice library which i use locally(quick plug: try the vagrant `xhprof` role). There are quite a few wfProfileIn/Out calls currently that generate sub-ms events. I am fairly certain these could be more accurately tracked with a php extension like xhprof.
I see wfProfileIn/Out data as still useful though, in one example for generating high level views for developers into what is happening(another plug, i wrote a visualization of wfProfileIn/Out data for the debug toolbar last weekend[1]). Also for the ?forceprofile=1 and ?forcetrace=1 options in production.
I think there is room for both our DIY and xhprof(but probably not xdebug in prod, its way too slow when generating traces).
Erik B.
[1] https://gerrit.wikimedia.org/r/#/c/104318/
On Tue, Dec 31, 2013 at 9:55 AM, Chad innocentkiller@gmail.com wrote:
I'm starting a new RFC to discuss ways we can improve our PHP profiling.
https://www.mediawiki.org/wiki/Requests_for_comment/Better_PHP_profiling
Please feel free to help expand and/or comment on the talk page if you've got ideas :)
-Chad _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
@Chad: should this be included in the straw poll for the architecture summit or is that too soon? D
On Tue, Dec 31, 2013 at 6:55 PM, Chad innocentkiller@gmail.com wrote:
I'm starting a new RFC to discuss ways we can improve our PHP profiling.
https://www.mediawiki.org/wiki/Requests_for_comment/Better_PHP_profiling
Please feel free to help expand and/or comment on the talk page if you've got ideas :)
-Chad _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
I don't think so. We chatted about it on IRC yesterday for a bit and I need to revise it significantly.
-Chad
On Fri, Jan 3, 2014 at 3:20 AM, Diederik van Liere dvanliere@wikimedia.orgwrote:
@Chad: should this be included in the straw poll for the architecture summit or is that too soon? D
On Tue, Dec 31, 2013 at 6:55 PM, Chad innocentkiller@gmail.com wrote:
I'm starting a new RFC to discuss ways we can improve our PHP profiling.
https://www.mediawiki.org/wiki/Requests_for_comment/Better_PHP_profiling
Please feel free to help expand and/or comment on the talk page if you've got ideas :)
-Chad _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Tue, Dec 31, 2013 at 9:55 AM, Chad innocentkiller@gmail.com wrote:
I'm starting a new RFC to discuss ways we can improve our PHP profiling.
https://www.mediawiki.org/wiki/Requests_for_comment/Better_PHP_profiling
Please feel free to help expand and/or comment on the talk page if you've got ideas :)
Apropos of wfProfileIn/Out:
"A tracing infrastructure that relies on active collaboration from application-level developers in order to function becomes extremely fragile, and is often broken due to instrumentation bugs or omissions, therefore violating the ubiquity requirement. This is especially important in a fast-paced development environment such as ours."
From "Dapper, a Large-Scale Distributed Systems Tracing Infrastructure"
http://research.google.com/pubs/pub36356.html
It's a really cool paper. Here's how Dapper makes it possible to instrument Google's distributed infrastructure:
* When a thread handles a traced control path, Dapper attaches a trace context to thread-local storage. A trace context is a small and easily copyable container of span attributes such as trace and span ids. * When computation is deferred or made asynchronous, most Google developers use a common control flow library to construct callbacks and schedule them in a thread pool or other executor. Dapper ensures that all such callbacks store the trace context of their creator, and this trace context is associated with the appropriate thread when the callback is invoked. In this way, the Dapper ids used for trace reconstruction are able to follow asynchronous control paths transparently. * Nearly all of Google’s inter-process communication is built around a single RPC framework with bindings in both C++ and Java. We have instrumented that framework to define spans around all RPCs. The span and trace ids are transmitted from client to server for traced RPCs.
(cf pg 4)
On Tue, Feb 18, 2014 at 7:02 AM, Ori Livneh ori@wikimedia.org wrote:
"A tracing infrastructure that relies on active collaboration from application-level developers in order to function becomes extremely fragile, and is often broken due to instrumentation bugs or omissions, therefore violating the ubiquity requirement. This is especially important in a fast-paced development environment such as ours."
I tend to agree. I'm really not a big fan of wfProfileIn/wfProfileOut. Among it's many issues:
- People often forget to call wfProfileOut (although this can be fixed by using ProfileSection) - It hurts readability (also can be fixed by ProfileSection, although only in cases where the entire function is being profiled) - It makes code completely dependent on MediaWiki, thus eliminating the possibility of separating code out into separate modules - It provides no more information than xhprof would (and yes, xhprof is meant for production use.
*-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2016 Major in Computer Science
wikitech-l@lists.wikimedia.org