-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
The Florida upload squid cluster started experiencing a problem with CPU usage maxed out and slow responses at about 16:45 UTC (45 minutes ago).
There is no problem with the internal file server, but images are loading very slowly via the web due to some kind of overload on the HTTP proxies.
We're poking at it but have not yet resolved the issue.
- -- brion vibber (brion @ wikimedia.org)
I noticed this but I also noticed it on bbc.co.uk so I thought it was me. In fact it probably is me and just coincidence!
On 09/10/2007, Brion Vibber brion@wikimedia.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
The Florida upload squid cluster started experiencing a problem with CPU usage maxed out and slow responses at about 16:45 UTC (45 minutes ago).
There is no problem with the internal file server, but images are loading very slowly via the web due to some kind of overload on the HTTP proxies.
We're poking at it but have not yet resolved the issue.
- -- brion vibber (brion @ wikimedia.org)
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHC7uVwRnhpk1wk44RAipzAKC3DSvTrLFcTPqTom0N7Y4mdQ7EfwCgqRmK 0Sh5KdWeE4EeKxIM9oo+6zU= =fkVb -----END PGP SIGNATURE-----
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Brion Vibber wrote:
The Florida upload squid cluster started experiencing a problem with CPU usage maxed out and slow responses at about 16:45 UTC (45 minutes ago).
There is no problem with the internal file server, but images are loading very slowly via the web due to some kind of overload on the HTTP proxies.
We're poking at it but have not yet resolved the issue.
We've disabled the IE 6 PNG workaround on en.wikipedia.org for now: http://en.wikipedia.org/wiki/MediaWiki_talk:Common.js#PNG_fix_disabled
The problem was manifesting primarily with loads of [[Image:Must left-click image again before saving.gif]], which was being substituted in for PNG images by the workaround code.
Backtraces seemed to be showing insanely long loops of httpHeaderGetEntry(), indicating possible memory corruption or other extreme weirdness.
We're not entirely sure how consistent it is or what exactly triggers it, but please don't turn it back on in the way it's currently coded without checking with us first.
-- brion vibber (brion @ wikimedia.org)
On 10/9/07, Brion Vibber brion@wikimedia.org wrote:
We're not entirely sure how consistent it is or what exactly triggers it, but please don't turn it back on in the way it's currently coded without checking with us first.
Brion should have also mentioned that this doesn't mean the PNG hack is itself broken (although it might be for other reasons ::shrugs::). The root cause is almost certainly something broken in Squid. The request to keep it disabled is precautionary until the underlying cause is understood and believed to be fixed.
wikitech-l@lists.wikimedia.org