Brion Vibber wrote:
Brion Vibber wrote:
Somebody badly broke Wiktionary etc by overwriting /home/common/php with files from /home/common/php-new. I'm trying to restore things.
I've got them working again, more or less.
At around May 31, 14:20, all the Wikipedias went down, displaying a PHP error. Shaihulud, Erik, Jeronim and myself were present in #mediawiki. My response was as follows. First I tried swapping the entire php-new directory for a backup in lazy-backups. This produced a 404 error on all wikipedias, so I swapped it back.
Next I enabled display_errors by hacking index.php. This showed that a file inclusion was failing due to the include path being set to /usr/local/apache/common/php instead of php-new. I investigated this error, finding that the correct ini_set statement was still present in CommonSettings.php. At a loss with the minutes of downtime wearing on, I moved the php directory to php.v12, and copied (cp -r) the php-new directory to php, then did a sync-common-all. This was intended as a stopgap measure, to give the script the proper files to include, while I tracked down the root cause of the problem. I was fully aware that it would break Wiktionary temporarily.
While it was synching, I spotted the root cause -- php-new/LocalSettings.php had been overwritten with a skeleton LocalSettings.php apparently from ab, which contained include("CommonSettings.php") instead of specifying a full path. Someone had probably mistakenly edited it via the symlinks in the htdocs directory.
After fixing this, I moved common/php to lazy-backups/php-new and moved common/php.v12 back to common/php, where it came from, then did a sync. Or at least, that's what I thought I did. If the files were overwritten, I guess I messed up something along the way. I didn't think to check if the Wiktionaries were working.
-- Tim Starling