Hi,
We're in the process of upgrading from 1.9 to 1.12 (I know I know I know...), and it seems that the including of extensions is taking a long time.
Profile from 1.9:
26.449 WebStart.php-conf 24.659 Setup.php 1.362 Setup.php-includes 2.117 Setup.php-misc1 1.020 Setup.php-memcached 0.570 Setup.php-SetupSession 0.117 Setup.php-globals 0.052 Setup.php-User 0.017 Setup.php-misc2 18.543 Setup.php-extensions
Seems fast, but 1.12:
594.761 WebStart.php-conf 0.012 WebStart.php-ob_start 207.409 Setup.php 50.838 Setup.php-includes 8.691 Setup.php-misc1 6.780 __autoload 9.632 Setup.php-memcached 0.685 __autoload 0.564 Setup.php-SetupSession 0.197 Setup.php-globals 0.046 Setup.php-User 0.023 Setup.php-misc2 128.051 Setup.php-extensions
This is pretty slow. It seems like for WebStart, having multiple inclusions like this:
require_once('extensions/wikihow/SpecialThankAuthors.php');
slows things down. Has something changed, or did we miss flipping a switch in the upgrade? What about Setup.php, does that have the same issue?
Thanks, Travis
On Tue, May 20, 2008 at 12:32 PM, Travis (wikiHow) travis@wikihow.com wrote:
Hi,
We're in the process of upgrading from 1.9 to 1.12 (I know I know I know...), and it seems that the including of extensions is taking a long time.
You may want to try asking Domas about this. I know there have been performance problems in loading extensions caused by extremely thorough localization in recent versions (e.g., Cite's internationalization file is 117 KB in 64 languages), but I'm pretty sure those hits have at least been deferred until the messages are needed.
Simetrical schreef:
On Tue, May 20, 2008 at 12:32 PM, Travis (wikiHow) travis@wikihow.com wrote:
Hi,
We're in the process of upgrading from 1.9 to 1.12 (I know I know I know...), and it seems that the including of extensions is taking a long time.
You may want to try asking Domas about this. I know there have been performance problems in loading extensions caused by extremely thorough localization in recent versions (e.g., Cite's internationalization file is 117 KB in 64 languages), but I'm pretty sure those hits have at least been deferred until the messages are needed.
You might wanna read serialized/README , it can improve performance in this regard. Also, upgrade your extensions if you haven't done so already. A few months ago, pretty much all extensions in our SVN repository were changed to only load their messages when they actually need them, rather than loading them for every request.
Roan Kattouw (Catrope)
I updated all of our extensions, and redesigned our custom extensions so that they lazy load messages and classes, it seemed to help, but didn't address Setup.php, and is still fairly slow:
152.432 WebStart.php-conf 0.011 WebStart.php-ob_start 236.918 Setup.php 53.863 Setup.php-includes 9.218 Setup.php-misc1 6.839 __autoload 9.986 Setup.php-memcached 0.713 __autoload 0.608 Setup.php-SetupSession 0.212 Setup.php-globals 0.053 Setup.php-User 0.029 Setup.php-misc2 153.725 Setup.php-extensions
Is there any reason why the Cite extension takes 60ms to load every time?
61.447 Setup.php-extensions-wfCite
Any other ideas? Is there any reason why this load time has jumped from 1.9 to 1.12?
Thanks, Travis
On Tue, May 20, 2008 at 1:50 PM, Roan Kattouw roan.kattouw@home.nl wrote:
Simetrical schreef:
On Tue, May 20, 2008 at 12:32 PM, Travis (wikiHow) travis@wikihow.com
wrote:
Hi,
We're in the process of upgrading from 1.9 to 1.12 (I know I know I know...), and it seems that the including of extensions is taking a long time.
You may want to try asking Domas about this. I know there have been performance problems in loading extensions caused by extremely thorough localization in recent versions (e.g., Cite's internationalization file is 117 KB in 64 languages), but I'm pretty sure those hits have at least been deferred until the messages are needed.
You might wanna read serialized/README , it can improve performance in this regard. Also, upgrade your extensions if you haven't done so already. A few months ago, pretty much all extensions in our SVN repository were changed to only load their messages when they actually need them, rather than loading them for every request.
Roan Kattouw (Catrope)
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Fri, Jun 6, 2008 at 3:41 PM, Travis (wikiHow) travis@wikihow.com wrote:
I updated all of our extensions, and redesigned our custom extensions so that they lazy load messages and classes, it seemed to help, but didn't address Setup.php, and is still fairly slow:
152.432 WebStart.php-conf 0.011 WebStart.php-ob_start 236.918 Setup.php 53.863 Setup.php-includes 9.218 Setup.php-misc1 6.839 __autoload 9.986 Setup.php-memcached 0.713 __autoload 0.608 Setup.php-SetupSession 0.212 Setup.php-globals 0.053 Setup.php-User 0.029 Setup.php-misc2 153.725 Setup.php-extensions
Is there any reason why the Cite extension takes 60ms to load every time?
61.447 Setup.php-extensions-wfCite
Any other ideas? Is there any reason why this load time has jumped from 1.9 to 1.12?
I don't know. Maybe you could try doing some more detailed profiling to see what the issue is? If there is an easily-fixable problem here, I'm sure we'd all like to fix it. If I'm reading this correctly, Setup.php is only taking 0.11 ms on Wikimedia, three orders of magnitude less:
http://noc.wikimedia.org/cgi-bin/ng/report.py?db=bigpage&sort=real&s...
Hi,
It seems part of our problem in Setup.php is the setting of function hooks. Just to clarify, this is the most efficient way to set a function hook, right?
$wgExtensionFunctions[] = 'wfWikiHowParserFunction_Setup';
function wfWikiHowParserFunction_Setup() { global $wgParser; $wgParser->setFunctionHook( 'ARTICLESSTARTED', 'articlesstarted'); $wgParser->setFunctionHook( 'PATROLCOUNT', 'patrolcount'); }
We have 2 different extensions that set up function hooks like this, it seems whichever one is executed first, takes 100ms, and the next one takes about 7ms, is there something that's initializing with wgParser on the first call? Is there anyway to cut down that first 100ms call?
Thanks Travis
On Fri, Jun 6, 2008 at 4:26 PM, Simetrical <Simetrical+wikilist@gmail.comSimetrical%2Bwikilist@gmail.com> wrote:
On Fri, Jun 6, 2008 at 3:41 PM, Travis (wikiHow) travis@wikihow.com wrote:
I updated all of our extensions, and redesigned our custom extensions so that they lazy load messages and classes, it seemed to help, but didn't address Setup.php, and is still fairly slow:
152.432 WebStart.php-conf 0.011 WebStart.php-ob_start 236.918 Setup.php 53.863 Setup.php-includes 9.218 Setup.php-misc1 6.839 __autoload 9.986 Setup.php-memcached 0.713 __autoload 0.608 Setup.php-SetupSession 0.212 Setup.php-globals 0.053 Setup.php-User 0.029 Setup.php-misc2 153.725 Setup.php-extensions
Is there any reason why the Cite extension takes 60ms to load every time?
61.447 Setup.php-extensions-wfCite
Any other ideas? Is there any reason why this load time has jumped from
1.9
to 1.12?
I don't know. Maybe you could try doing some more detailed profiling to see what the issue is? If there is an easily-fixable problem here, I'm sure we'd all like to fix it. If I'm reading this correctly, Setup.php is only taking 0.11 ms on Wikimedia, three orders of magnitude less:
http://noc.wikimedia.org/cgi-bin/ng/report.py?db=bigpage&sort=real&s...
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Travis (wikiHow) wrote:
It seems part of our problem in Setup.php is the setting of function hooks. Just to clarify, this is the most efficient way to set a function hook, right?
$wgExtensionFunctions[] = 'wfWikiHowParserFunction_Setup';
function wfWikiHowParserFunction_Setup() { global $wgParser; $wgParser->setFunctionHook( 'ARTICLESSTARTED', 'articlesstarted'); $wgParser->setFunctionHook( 'PATROLCOUNT', 'patrolcount'); }
This will force $wgParser to be unstubbed at setup time. This isn't the end of the world, but it will have a couple of effects:
* If you don't have a working PHP opcode cache, this means that the big Parser class gets loaded whether you need it or not
* Actual parser object initialization happens whether you need it or not
* Parser initialization will be accounted in the extension setup phase in your profiling, which may skew the visible results.
In practice, this probably won't cause too much trouble. The parser is going to get unstubbed on just about any request that outputs HTML; it'll get used for UI message transformations even if you're outputting a page from the parser cache.
But for 304 "not changed" results, and potentially for some API hits and stuff, it might shave off a few milliseconds to be able to skip it.
To do that, you want to call setFunctionHook() from a parser first call init hook, as currently done by a lot of our extensions, as mentioned in my last post in this thread.
- -- brion
Great, that makes total sense and seems to have worked well. I think it looks like for article views, 1.12 is performing better than 1.9 now, thanks for the help.
For opcode caching, we're using eaccelerator, should we be using something else? What does WP use?
Also, I noticed that the PHP sapi name for WP is just apache, while ours is apache2handler. Could there be a difference in terms of performance between the 2? How do you go about creating just an apache module?
Thanks, Travis
On Mon, Jun 9, 2008 at 11:52 AM, Brion Vibber brion@wikimedia.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Travis (wikiHow) wrote:
It seems part of our problem in Setup.php is the setting of function
hooks.
Just to clarify, this is the most efficient way to set a function hook, right?
$wgExtensionFunctions[] = 'wfWikiHowParserFunction_Setup';
function wfWikiHowParserFunction_Setup() { global $wgParser; $wgParser->setFunctionHook( 'ARTICLESSTARTED', 'articlesstarted'); $wgParser->setFunctionHook( 'PATROLCOUNT', 'patrolcount'); }
This will force $wgParser to be unstubbed at setup time. This isn't the end of the world, but it will have a couple of effects:
- If you don't have a working PHP opcode cache, this means that the big
Parser class gets loaded whether you need it or not
Actual parser object initialization happens whether you need it or not
Parser initialization will be accounted in the extension setup phase
in your profiling, which may skew the visible results.
In practice, this probably won't cause too much trouble. The parser is going to get unstubbed on just about any request that outputs HTML; it'll get used for UI message transformations even if you're outputting a page from the parser cache.
But for 304 "not changed" results, and potentially for some API hits and stuff, it might shave off a few milliseconds to be able to skip it.
To do that, you want to call setFunctionHook() from a parser first call init hook, as currently done by a lot of our extensions, as mentioned in my last post in this thread.
- -- brion
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkhNUa4ACgkQwRnhpk1wk45pugCgs6ESM+jWNVsJFRQUxlK4og0B GZgAn3bIJwUZbjnHaZCwekCuvNMoDXr0 =Vnq+ -----END PGP SIGNATURE-----
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Travis (wikiHow) wrote:
Great, that makes total sense and seems to have worked well. I think it looks like for article views, 1.12 is performing better than 1.9 now, thanks for the help.
For opcode caching, we're using eaccelerator, should we be using something else? What does WP use?
We use APC; last I looked eaccelerator was pretty flake on PHP 5, especially 5.1 and later, but that might have changed. YMMV. :)
Also, I noticed that the PHP sapi name for WP is just apache, while ours is apache2handler. Could there be a difference in terms of performance between the 2? How do you go about creating just an apache module?
As I recall...
apache = Apache 1.3 module apache2 = Apache 2.0 module apache2handler = Apache 2.0 module using a different Apache API
There's probably not a huge difference in performance, but again YMMV. :)
If in doubt, try them out.
- -- brion
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Travis (wikiHow) wrote:
I updated all of our extensions, and redesigned our custom extensions so that they lazy load messages and classes, it seemed to help, but didn't address Setup.php, and is still fairly slow:
152.432 WebStart.php-conf 0.011 WebStart.php-ob_start 236.918 Setup.php 53.863 Setup.php-includes 9.218 Setup.php-misc1 6.839 __autoload 9.986 Setup.php-memcached 0.713 __autoload 0.608 Setup.php-SetupSession 0.212 Setup.php-globals 0.053 Setup.php-User 0.029 Setup.php-misc2 153.725 Setup.php-extensions
Is there any reason why the Cite extension takes 60ms to load every time?
61.447 Setup.php-extensions-wfCite
Cite is triggering an unstub of the parser for its initialization (and thus a bunch of other code gets loaded, making the profiling lopsided).
I've just broken out the setHook() calls to ParserFirstCallInit hooks for Cite and several other extensions we use (in r35980). This has already been done in a number of other extensions.
- -- brion vibber (brion @ wikimedia.org)
wikitech-l@lists.wikimedia.org