Brion Vibber wrote:
Brion Vibber wrote:
Somebody badly broke Wiktionary etc by
overwriting /home/common/php
with files from /home/common/php-new. I'm trying to restore things.
I've got them working again, more or less.
At around May 31, 14:20, all the Wikipedias went down, displaying a PHP
error. Shaihulud, Erik, Jeronim and myself were present in #mediawiki.
My response was as follows. First I tried swapping the entire php-new
directory for a backup in lazy-backups. This produced a 404 error on all
wikipedias, so I swapped it back.
Next I enabled display_errors by hacking index.php. This showed that a
file inclusion was failing due to the include path being set to
/usr/local/apache/common/php instead of php-new. I investigated this
error, finding that the correct ini_set statement was still present in
CommonSettings.php. At a loss with the minutes of downtime wearing on, I
moved the php directory to php.v12, and copied (cp -r) the php-new
directory to php, then did a sync-common-all. This was intended as a
stopgap measure, to give the script the proper files to include, while I
tracked down the root cause of the problem. I was fully aware that it
would break Wiktionary temporarily.
While it was synching, I spotted the root cause --
php-new/LocalSettings.php had been overwritten with a skeleton
LocalSettings.php apparently from ab, which contained
include("CommonSettings.php") instead of specifying a full path. Someone
had probably mistakenly edited it via the symlinks in the htdocs directory.
After fixing this, I moved common/php to lazy-backups/php-new and moved
common/php.v12 back to common/php, where it came from, then did a sync.
Or at least, that's what I thought I did. If the files were overwritten,
I guess I messed up something along the way. I didn't think to check if
the Wiktionaries were working.
-- Tim Starling