As a precaution I've lowered the memory usage on MySQL. Below is the my.cnf currently used (minus comments). I reduced the key buffer and InnoDB buffer pool each from 1G to 768M, and the max connections from 650 (which allowed for generous attempts at double connections if every apache process were running the wiki and had a broken persistent connection, a new connection on top, and maybe some extra) to 300. (Apache is now set to allow a max of 120 connections on larousse and 150 on pliny; this leaves only a little bit extra if all those connections get used).

The maximum possible memory usage should be roughly 4MB*300 connections plus 768M plus another 768M? Under 3 gigs, in theory, of the 4G geoffrin has. Under the previous configuration a complete stall-out which used the full memory maxima would have gone into swap which it may not handle well; this may be responsible for a signal 11 mysqld crash a few hours ago and the more recent hang.

Unfortunately neither time I had a vmstat sitting around.

[client] port = 3306 socket = /tmp/mysql.sock

[mysqld] port = 3306 socket = /tmp/mysql.sock skip-locking key_buffer = 768M max_allowed_packet = 1M table_cache = 1000 sort_buffer_size = 2M read_buffer_size = 2M myisam_sort_buffer_size = 64M thread_cache = 8 query_cache_size = 32M thread_concurrency = 8

set-variable = max_connections=300 log-bin set-variable = ft_min_word_len=2 server-id = 1 tmpdir = /var/tmp/

innodb_buffer_pool_size = 768M innodb_log_file_size = 128M innodb_log_buffer_size = 8M innodb_flush_log_at_trx_commit = 2

[mysqldump] quick max_allowed_packet = 16M

[mysql] no-auto-rehash

[isamchk] key_buffer = 256M sort_buffer_size = 256M read_buffer = 2M write_buffer = 2M

[myisamchk] key_buffer = 256M sort_buffer_size = 256M read_buffer = 2M write_buffer = 2M

[mysqlhotcopy] interactive-timeout

-- brion vibber (brion @ pobox.com)