I'm looking again at my log files, comparing three days: - Thursday May 9, before Jimmy's change (old talk links) - Friday May 10, after Jimmy's change (no talk links) - Tuesday May 14, with the new version (new talk links)

The overall performance was best on Friday. It is now getting worse again, albeit not as bad as Thursday. The new version is a real improvement over the old talk links, but we aren't quite done yet.

The number of OK responses (HTTP status code 200) which take absurdly long (longer than 60 seconds) is still very high (3-30 %), with the Main Page of the English Wikipedia being the main exception (0 %).

Response times above some limit (say, 30, 60 or 120 seconds) can be defined as absurdly long, because the user will have left for other websites and is no longer waiting for the response. Instead of spending more system resources (CPU cycles and allocated memory) on these requests, it would be better to set a hard time out (in PHP or Apache) and return an error message that says "sorry for the delay". This would free up system resources that can be better used to serve other requests.

In http://www.php.net/manual/en/function.set-time-limit.php, the PHP function set_time_limit() is said to have a default of 30 seconds, unless the configuration file has defined max_execution_time. Will calling this function set the time limit for the current request only, or set a permanent value for the server? What happens when PHP execution times out? Is the connection to the client abruptly closed? Or is an error message returned? Does an error message appear in the log file? I haven't seen any timeouts of this kind.

However, even if a PHP execution timeout is set, the limit will not include time that the request spent waiting to start execution. This wait could happen in the UNIX socket listen backlog, waiting for the connection to be accepted, or inside Apache, waiting for a child process to become available. Increasing the value of a parameter like ListenBacklog (in Apache httpd.conf) is not necessarily a solution, because this will only keep more requests in queue, increasing overall response time. Instead, the problem should be fixed at the exit end of the queue. The key to better performance is keeping the server fast and queues short, getting things done.

Here are some pages on Apache performance issues:

- Hints on Running a High-Performance Web Server, http://httpd.apache.org/docs/misc/perf.html - Apache Performance Notes, http://httpd.apache.org/docs/misc/perf-tuning.html - Professional Apache, chapter 8, http://www.devshed.com/Talk/Books/ProApache/ - Tuning Your Apache Web Server, http://dcb.sun.com/practices/howtos/tuning_apache.jsp - Linux HTTP Benchmarking HOWTO, http://www.xenoclast.org/doc/benchmark/HTTP-benchmarking-HOWTO/