As OQ mentioned, if your MaxClients in Apache is set too high when you get
a burst of connections like this you'll eat up all your RAM and begin
swapping which kills performance real quick. If you don't want it to fail
in this way you can tune your MaxClients to a point where even with a burst
you don't use up all your RAM at the expense of dropping/refusing
connections.
As a rough guess at what to set MaxClients to use "top" and look at the
difference between the RES and SHR columns of all httpd processes. I
believe this is roughly the amount of non-shared memory each of the child
Apache processes are using (for example, I'm averaging 10 MB per process).
Take the maximum amount of RAM you want Apache to use and divide it by this
memory to get a rough number for MaxClients. You can monitor memory usage
and adjust this as needed or through load testing.
Of course, this is a solution to the usage of swap and not the actual issue
you are having. I would continue to look at what is causing the surge in
requests. Is it a DoS of some sort (either accidental or on purpose) or is
some part of the server stalling which is causing requests to pile up and
overflow? For example, if the database is having an issue (a bunch of long
queries) then all Apache requests will start piling up until you hit swap
or the database issue resolves itself.
Along these lines I would suggest some sort of monitoring/logging service
like Zabbix or Nagois (to name just two, there are many similar options to
choose from). This helps you on two fronts: 1) Logging of Parameters and 2)
Service Monitoring. Trying to diagnose issues after the fact can be
difficult/impossible and with such a service you have a record of many
things to help you pinpoint, or at least narrow down, the root cause of the
issue. Monitoring is also invaluable as you can be emailed/texted when the
issue actually happens (or is just starting) rather than 5 minutes after
the website has begun timing out for everyone.
On 29 October 2012 19:14, David Gerard <dgerard(a)gmail.com> wrote:
So,
rationalwiki.org has been *much* faster and more
usable with a
coupla squids and a load-balancer in front of the
Apache/Lucene/database node. (We could probably cope with just one
squid, but Trent wanted to experiment.) The nodes are all Ubuntu 10.04
Linodes, the software manually kept up to date.
....
--
Dave Humphrey -- dave(a)uesp.net
Founder/Server Admin of the Unofficial Elder Scrolls Pages --
www.uesp.net