I suspect the problem is somewhere in MediaWiki. If knew where I'd be
posting it here.
- I've stripped away other things on the server. Pretty much the only
thing left is the most recent version of MediaWiki (1.23.3) and Debian
(stable) packages.
- I ran 'debsums' to check whether any of the Debian packages are corrupt.
- I am not aware of any LAMP bugs in Debian.
The MediaWiki install is near fresh. I did a manual upgrade for
version 1.23.1 and then used the patches to get to 1.23.3. The
upgrade from 1.23.1 to 1.23.2 had a bug in it-- but I think that was
inconsequential. In any case, that upgrade was done July 30, several
weeks preceding the problems.
Here is a snippet from 'ps aux':
-----
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
...
www-data 10518 0.0 2.8 291376 57052 ? S 20:39 0:03
/usr/sbin/apache2 -k start
www-data 10567 0.0 2.5 288076 51480 ? S 20:42 0:02
/usr/sbin/apache2 -k start
www-data 10568 0.0 2.6 290496 54140 ? S 20:42 0:04
/usr/sbin/apache2 -k start
www-data 10569 0.0 2.5 288820 51840 ? S 20:42 0:02
/usr/sbin/apache2 -k start
www-data 10570 0.0 2.6 291384 53036 ? S 20:42 0:02
/usr/sbin/apache2 -k start
www-data 10571 0.0 2.9 293316 58812 ? S 20:42 0:05
/usr/sbin/apache2 -k start
www-data 10572 0.0 2.7 290976 55360 ? S 20:42 0:04
/usr/sbin/apache2 -k start
www-data 10573 0.0 2.7 290888 55820 ? S 20:42 0:05
/usr/sbin/apache2 -k start
www-data 10649 0.0 2.7 290728 54860 ? S 20:53 0:05
/usr/sbin/apache2 -k start
www-data 10650 0.0 2.7 293000 55924 ? S 20:53 0:04
/usr/sbin/apache2 -k start
...
-----
The memory seems to be going into apache2.
Earlier this morning I rebooted the server. About three hours later the
images were gone again. After rebooting this morning... the server has
been working okay (so far about 13 hours). I don't understand the cycle:
why it breaks in 3 hours... or goes for more than 12.
Some ideas I have:
1. Running a 'diff' of my active server against the tarball for the
most recent MediaWiki-- to look for corrupted files.
2. At this point I am thinking about the extensions and doing a fresh
install of MediaWiki... with the hope that will resolve things.
3. I have bimonthly images going back a year-- I can dump the database,
rollback and grab the latest database changes.
My inclination is to try #2. It isn't that hard to do... if
it works I can be pretty sure it was MediaWiki... and it solves the problem
I have at the moment.
Any thoughts/ideas are welcome.
Michael