Just curious about ways to maybe save bandwidth for little mediawiki installations as well as wikipedia:
Have the logos at the bottom of the screen, background images use the central wikimedia ones ("hotlinked") instead of the ones that come from the distribution .tar. And on the central wikimedia ones, have Expires: Jan 1 2036 etc. HTTP header. Result: one copy cached forever on the whole Internet? If one day regret Expires 2036, then just change the filename when needed. Or maybe expires 1 year form now, etc.
Maybe also somewhat for .css.
On 3/11/07, Dan Jacobson jidanni@jidanni.org wrote:
Just curious about ways to maybe save bandwidth for little mediawiki installations as well as wikipedia:
Have the logos at the bottom of the screen, background images use the central wikimedia ones ("hotlinked") instead of the ones that come from the distribution .tar. And on the central wikimedia ones, have Expires: Jan 1 2036 etc. HTTP header. Result: one copy cached forever on the whole Internet? If one day regret Expires 2036, then just change the filename when needed. Or maybe expires 1 year form now, etc.
Maybe also somewhat for .css.
That would be great if a) the Wikimedia Foundation had lots of bandwidth and Squid resources to spare, and b) it never ever ever would ever not have lots of bandwidth and Squid resources to spare (since packaging things this way would be tantamount to a promise never to remove them). Also, it'll be fun when every MediaWiki install on the Internet looks ugly every time the cluster goes down, and I'm sure it would be viewed as very professional to have a third-party wiki with zero users take ten seconds to fully load a cache miss at peak hours.
It wouldn't work at all for CSS, since that changes on a regular basis.
Not to mention HTTPS - which would cause "This page contains both secure and non-secure elements." dialogs.
On 3/11/07, Simetrical Simetrical+wikilist@gmail.com wrote:
On 3/11/07, Dan Jacobson jidanni@jidanni.org wrote:
Just curious about ways to maybe save bandwidth for little mediawiki installations as well as wikipedia:
Have the logos at the bottom of the screen, background images use the central wikimedia ones ("hotlinked") instead of the ones that come from the distribution .tar. And on the central wikimedia ones, have Expires: Jan 1 2036 etc. HTTP header. Result: one copy cached forever on the whole Internet? If one day regret Expires 2036, then just change the filename when needed. Or maybe expires 1 year form now, etc.
Maybe also somewhat for .css.
That would be great if a) the Wikimedia Foundation had lots of bandwidth and Squid resources to spare, and b) it never ever ever would ever not have lots of bandwidth and Squid resources to spare (since packaging things this way would be tantamount to a promise never to remove them). Also, it'll be fun when every MediaWiki install on the Internet looks ugly every time the cluster goes down, and I'm sure it would be viewed as very professional to have a third-party wiki with zero users take ten seconds to fully load a cache miss at peak hours.
It wouldn't work at all for CSS, since that changes on a regular basis.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 3/12/07, Dan Jacobson jidanni@jidanni.org wrote:
Just curious about ways to maybe save bandwidth for little mediawiki installations as well as wikipedia:
How does this save Wikipedia bandwidth?
Steve
Say, do the boilerplate images always present on every Mediawiki page have enough Expires and/or etc. headers to keep them cached around the network for a long time? (Can't tell here offline with images turned off anyway). Seems like optimizing that would need messing with the apache etc. configuration as it is beyond what one can do with php as they are <img src=...>. I.e., well at least we will send "304 not modified", as we (standard Mediawiki installation) are not going to go further to eliminate them asking in the first place... but as you can see, what do I know.
On 3/15/07, Dan Jacobson jidanni@jidanni.org wrote:
Say, do the boilerplate images always present on every Mediawiki page have enough Expires and/or etc. headers to keep them cached around the network for a long time? (Can't tell here offline with images turned off anyway). Seems like optimizing that would need messing with the apache etc. configuration as it is beyond what one can do with php as they are <img src=...>.
WMF images are generally served with instructions to cache for 30 days, IIRC. Nevertheless, a fairly large percentage of hits are cache misses for the images, is my impression. Cache-Control can specify maximum cache ages, but not minimum.
Simetrical wrote:
WMF images are generally served with instructions to cache for 30 days, IIRC.
30 days for /skins-1.5/common/images/ I don't get Expired nor Cache-Control for upload.wikimedia.org, though :S
Cache-Control can specify maximum cache ages, but not minimum.
No matter what you tell them, the minimum will always be decided by the client. :-)
Nevertheless, a fairly large percentage of hits are cache misses for the images, is my impression.
I just got this:
X-Cache: MISS from sq6.wikimedia.org X-Cache-Lookup: MISS from sq6.wikimedia.org:80 X-Cache: HIT from knsq8.knams.wikimedia.org X-Cache-Lookup: HIT from knsq8.knams.wikimedia.org:80 X-Cache: HIT from knsq15.knams.wikimedia.org X-Cache-Lookup: HIT from knsq15.knams.wikimedia.org:80 X-Cache: HIT from knsq13.knams.wikimedia.org X-Cache-Lookup: HIT from knsq13.knams.wikimedia.org:80 X-Cache: HIT from knsq10.knams.wikimedia.org X-Cache-Lookup: HIT from knsq10.knams.wikimedia.org:80 X-Cache: HIT from knsq14.knams.wikimedia.org X-Cache-Lookup: HIT from knsq14.knams.wikimedia.org:80 X-Cache: HIT from knsq12.knams.wikimedia.org X-Cache-Lookup: HIT from knsq12.knams.wikimedia.org:80 X-Cache: HIT from knsq11.knams.wikimedia.org X-Cache-Lookup: HIT from knsq11.knams.wikimedia.org:80 X-Cache: HIT from knsq13.knams.wikimedia.org X-Cache-Lookup: HIT from knsq13.knams.wikimedia.org:80 X-Cache: HIT from knsq15.knams.wikimedia.org X-Cache-Lookup: HIT from knsq15.knams.wikimedia.org:80 X-Cache: HIT from knsq14.knams.wikimedia.org X-Cache-Lookup: HIT from knsq14.knams.wikimedia.org:80 X-Cache: HIT from knsq10.knams.wikimedia.org X-Cache-Lookup: HIT from knsq10.knams.wikimedia.org:80 X-Cache: HIT from knsq9.knams.wikimedia.org X-Cache-Lookup: HIT from knsq9.knams.wikimedia.org:80 X-Cache: HIT from knsq14.knams.wikimedia.org X-Cache-Lookup: HIT from knsq14.knams.wikimedia.org:80 X-Cache: HIT from knsq15.knams.wikimedia.org X-Cache-Lookup: HIT from knsq15.knams.wikimedia.org:80 X-Cache: HIT from knsq8.knams.wikimedia.org X-Cache-Lookup: HIT from knsq8.knams.wikimedia.org:80 X-Cache: HIT from knsq13.knams.wikimedia.org X-Cache-Lookup: HIT from knsq13.knams.wikimedia.org:80 X-Cache: HIT from knsq12.knams.wikimedia.org X-Cache-Lookup: HIT from knsq12.knams.wikimedia.org:80 X-Cache: HIT from knsq11.knams.wikimedia.org X-Cache-Lookup: HIT from knsq11.knams.wikimedia.org:80 X-Cache: HIT from knsq10.knams.wikimedia.org X-Cache-Lookup: HIT from knsq10.knams.wikimedia.org:80 X-Cache: HIT from knsq9.knams.wikimedia.org X-Cache-Lookup: HIT from knsq9.knams.wikimedia.org:80 Age: 121213 X-Cache: HIT from knsq13.knams.wikimedia.org X-Cache-Lookup: HIT from knsq13.knams.wikimedia.org:3128 X-Cache: MISS from knsq9.knams.wikimedia.org X-Cache-Lookup: MISS from knsq9.knams.wikimedia.org:80 Via: 1.0 sq6.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq8.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq15.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq13.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq10.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq14.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq12.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq11.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq13.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq15.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq14.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq10.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq9.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq14.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq15.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq8.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq13.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq12.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq11.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq10.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq9.knams.wikimedia.org:80 (squid/2.6.STABLE9), 1.0 knsq13.knams.wikimedia.org:3128 (squid/2.6.STABLE 9), 1.0 knsq9.knams.wikimedia.org:80 (squid/2.6.STABLE9)
On 3/16/07, Platonides Platonides@gmail.com wrote:
I just got this:
X-Cache: MISS from sq6.wikimedia.org X-Cache-Lookup: MISS from sq6.wikimedia.org:80 X-Cache: HIT from knsq8.knams.wikimedia.org X-Cache-Lookup: HIT from knsq8.knams.wikimedia.org:80 X-Cache: HIT from knsq15.knams.wikimedia.org X-Cache-Lookup: HIT from knsq15.knams.wikimedia.org:80 X-Cache: HIT from knsq13.knams.wikimedia.org . . .
I meant *client* cache misses. Which is what this thread was about. The Squids are surely going to have all the interface images cached, at least. (Assuming the Squids cache those and not just pages.)
On 11/03/07, Dan Jacobson jidanni@jidanni.org wrote:
Just curious about ways to maybe save bandwidth for little mediawiki installations as well as wikipedia:
MediaWiki isn't designed to run on crappy old hardware serving a few users. It's a wiki engine designed to scale to meet the needs of millions of users; billions of requests, on modern hardware.
There is a lower limit at which MediaWiki stops being the best choice. Allowing other people to hotlink images from our sites is not an option. If your web server can't handle the load of a few images, then you probably need to upgrade from a PDP-11 running lighty on 56k.
Rob Church
wikitech-l@lists.wikimedia.org