Brion Vibber: I noticed that the 100% CPU occurs after index.php has finished, and after return from the output callback. My guess is the call to header() in the output callback saveToFileCache() is not safe. This is writing to the buffer that the output callback is processing. if( $this->useGzip() ) { if( wfClientAcceptsGzip() ) { header( 'Content-Encoding: gzip' ); Perhaps this confuses PHP. Also, I guess this header doesn't actually make it into the headers, so perhaps the gzipped data is confusing something down the line. -- Regards / Groeten, http://www.leren.nl René Pijlman http://www.applinet.nl

Rene Pijlman: BTW it works fine now with $wgUseGzip = false; in LocalSettings.php. The generic caching problem is fixed by Brion's patch, a compressed caching problem remains. Correcting myself... I forgot that wget by default sends a different Accept-Encoding header. When I run it with: wget -S --header='Accept-Encoding: gzip, deflate' url the headers look OK: 1 HTTP/1.1 200 OK 2 Date: Sun, 08 Aug 2004 14:39:31 GMT 3 Server: Apache/1.3.26 (Unix) Debian GNU/Linux mod_python/2.7.8 Python/2.1.3 PHP/4.1.2 4 X-Powered-By: PHP/4.1.2 5 Vary: Accept-Encoding 6 Expires: -1 7 Cache-Control: private, must-revalidate, max-age=0 8 Last-modified: Sat, 7 Aug 2004 22:51:25 GMT 9 Content-Encoding: gzip 10 Content-Length: 1803 11 Keep-Alive: timeout=15, max=100 12 Connection: Keep-Alive 13 Content-Type: text/html; charset=iso-8859-1 14 Content-Language: nl ... and wget stores the gzipped data in a file (that's correct I guess). With zcat it looks fine. The question remains: why don't Firefox and IE uncompress the data before rendering... -- Regards / Groeten, http://www.leren.nl René Pijlman http://www.applinet.nl

Brion Vibber: Oh waitaminute, you're referring to this I guess: if( !ini_get( 'zlib.output_compression' ) ) ob_start( 'ob_gzhandler' ); I completely overlooked that. Indeed, when I remove this line and configure file caching like this: $wgUseFileCache = true; $wgFileCacheDirectory = "/home/rene/projects/carriere/cache"; $wgShowIPinHeader = false; $wgUseGzip = true; ... it works fine. Files in the cache are compressed and both Firefox and IE render pages correctly. Thanks again for your help Brion. If I'm not mistaken the cause of the problem was that both ob_gzhandler and the file cache were compressing the output, so it was compressed two times, which would explain the problem I saw. May I suggest to fix this in the code, to make configuration easier? I'd say that when PHP provides the mechanism for output compression over the wire, there's no reason to duplicate this mechanism in the file cache. You might as well completely remove compression from the file cache and cache uncompressed on disk. At the cost of some extra disk space this will improve performance of the file cache (no CPU cycles for (un)compression) and simplify the code. This will neatly separate caching from compression. Would it help if I implement and test this and submit a patch? -- Regards / Groeten, http://www.leren.nl René Pijlman http://www.applinet.nl

Brion Vibber: Hmm, I assumed this was intended to compress the data that is sent to the client. Well OK, in that case I suggest to make the file cache work correctly in all cases, e.g. by sending uncompressed data to the output buffer when the PHP lib will take care of compression. The wfClientAcceptsGzip() branches of the code seem to duplicate the functionality of ob_gzhandler, and can be removed without loss of functionality. An optimization would be to solve it the other way around: decide per request to not compress with ob_gzhandler when the file is available in compressed form in the file cache. -- Regards / Groeten, http://www.leren.nl René Pijlman http://www.applinet.nl

Brion Vibber: Another problem is zlib.output_compression which is set in php.ini and cannot be turned off at the scripting level (according to a comment in the docs). And I guess most users will want to enable it to compress all requests, including requests that cannot be served from the file cache. So I don't think an implementation which serves compressed cached files without the overhead of uncompress/compress is worth the effort. I have attached a patch which fixes the compressed file cache, by always sending uncompressed data to the output buffer, leaving compression up to zlib.output_compression or ob_gzhandler. The effect of the patch is that the file cache can be enabled with $wgUseGzip set to true or false. $wgUseGzip decides if the files stored in the cache are compressed or not. But I think there should be a comment in the documentation that setting $wgUseGzip to true makes little sense. The idead of the cache is to spend some disk space to reduce CPU-cycles, so why would you then want to spend CPU-cycles to reduce that disk space? I also suggest to change this in DefaultSettings.php: # We can serve pages compressed in order to save bandwidth, # but this will increase CPU usage. # Requires zlib support enabled in PHP. $wgUseGzip = function_exists( 'gzencode' ); to: # Should the file cache be compressed, in order to save disk # space. This will increase CPU usage. # Requires zlib support enabled in PHP. To enable, change # this line to: # $wgUseGzip = function_exists('gzencode'); $wgUseGzip = false; With this patch applied, wfClientAcceptsGzip() will no longer be used. -- Regards / Groeten, http://www.leren.nl René Pijlman http://www.applinet.nl