-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I took a quick peek at the sampled squid log and found that CSS and JS files together are eating a lot of bandwidth; together they make up about 20% of what's served:
https://wikitech.leuksman.com/view/Squid_bandwidth_breakdown
(May be inaccurate due to coding mistakes in my counter or weird dupe caching effects.)
It should be possible to serve these files compressed through Apache with mod_gzip set up, which should squish them by probably 2/3.
- -- brion vibber (brion @ pobox.com / brion @ wikimedia.org)
I took a quick peek at the sampled squid log and found that CSS and JS files together are eating a lot of bandwidth; together they make up about 20% of what's served:
https://wikitech.leuksman.com/view/Squid_bandwidth_breakdown
(May be inaccurate due to coding mistakes in my counter or weird dupe caching effects.)
Please forgive me if this is a dumb question, but if I check the headers returned for two successive requests, like so:
====================================================== root@bling:/var/www/hosts/mediawiki/wiki# curl --silent --include --head http://en.wikipedia.org/skins-1.5/monobook/main.css?55 HTTP/1.0 200 OK Date: Wed, 07 Feb 2007 02:56:31 GMT Server: Apache Cache-Control: max-age=2592000 Expires: Fri, 09 Mar 2007 02:56:31 GMT Last-Modified: Tue, 06 Feb 2007 20:04:40 GMT ETag: "60874b-709d-45c8df58" Accept-Ranges: bytes Content-Length: 28829 Content-Type: text/css Age: 2 X-Cache: HIT from sq30.wikimedia.org X-Cache-Lookup: HIT from sq30.wikimedia.org:80 Via: 1.0 sq30.wikimedia.org:80 (squid/2.6.STABLE9) Connection: close
root@bling:/var/www/hosts/mediawiki/wiki# curl --silent --include --head http://en.wikipedia.org/skins-1.5/monobook/main.css?55 HTTP/1.0 200 OK Date: Wed, 07 Feb 2007 02:56:26 GMT Server: Apache Cache-Control: max-age=2592000 Expires: Fri, 09 Mar 2007 02:56:26 GMT Last-Modified: Tue, 06 Feb 2007 20:04:40 GMT ETag: "15c02de-709d-45c8df58" Accept-Ranges: bytes Content-Length: 28829 Content-Type: text/css Age: 9 X-Cache: HIT from sq20.wikimedia.org X-Cache-Lookup: HIT from sq20.wikimedia.org:80 Via: 1.0 sq20.wikimedia.org:80 (squid/2.6.STABLE9) Connection: close
root@bling:/var/www/hosts/mediawiki/wiki# ======================================================
... then I have two questions:
1) Does it matter that the ETag varies between successive requests? Reason I ask is that the http://www.web-caching.com/mnot_tutorial/how.html page says: "HTTP 1.1 introduced a new kind of validator called the ETag. ETags are unique identifiers that are generated by the server and changed every time the object does. Because the server controls how the ETag is generated, caches can be surer that if the ETag matches when they make a If-None-Match request, the object really is the same."
I.e. if the ETag changes between requests, as it did in the above example, could that make requesters think that the object has changed too, thus reducing caching?
2) Would it help using "Cache-Control: max-age=2592000, public" instead of "Cache-Control: max-age=2592000" ? Public is defined as "marks the response as cacheable, even if it would normally be uncacheable. For instance, if your pages are authenticated, the public directive makes them cacheable." I.e. I'm not sure if the Wikipedia cookie is being treated as authentication for the purposes of this definition, but if it is, caching the site-wide CSS or JS seems unlikely to hurt (since it really is "public") - but obviously caching the user-specific CSS or JS would be bad.
It should be possible to serve these files compressed through Apache with mod_gzip set up, which should squish them by probably 2/3.
Last time I looked mod_gzip seemed to be losing favour somewhat - the new "in" compression method for apache2 seems to be mod_deflate ( http://httpd.apache.org/docs/2.0/mod/mod_deflate.html ), partially because it's an apache module, rather than a 3rd-party module; however it got a few percentage points less efficient compression than mod_gzip, but the suggestion was that it caused less CPU load to do its compression. It was about a year ago that I looked at this stuff though, so the state of play may have changed since.
All the best, Nick.
ETag: "60874b-709d-45c8df58"
[...snip...]
ETag: "15c02de-709d-45c8df58"
[...snip...]
- Does it matter that the ETag varies between successive requests? Reason I ask is that
the http://www.web-caching.com/mnot_tutorial/how.html page says: "HTTP 1.1 introduced a new kind of validator called the ETag. ETags are unique identifiers that are generated by the server and changed every time the object does. Because the server controls how the ETag is generated, caches can be surer that if the ETag matches when they make a If-None-Match request, the object really is the same."
I.e. if the ETag changes between requests, as it did in the above example, could that make requesters think that the object has changed too, thus reducing caching?
Just found how to stop this - The ETag comes from 3 sources of information - the inode, the file size, and the file modified time.
That's why these two are different yet similar:
ETag: "60874b-709d-45c8df58" ETag: "15c02de-709d-45c8df58"
I.e. the format currently is ETag: "<inode value>-<date value>-<file size value>". [which is the apache2 default, I think].
The inodes differ for the different systems, but the date and the file values are the same, as you can see.
The solution therefore is to add this line to the cluster's apache2.conf: ------------------------- FileETag MTime Size -------------------------
Then the result will be: ETag: "709d-45c8df58" for all boxes serving this example file.
All the best, Nick.
Nick Jenkins wrote:
I'm not sure if the Wikipedia cookie is being treated as authentication for the purposes of this definition, but if it is, caching the site-wide CSS or JS seems unlikely to hurt (since it really is "public") - but obviously caching the user-specific CSS or JS would be bad.
Why? User-specific CSS and JS is still public.
Matthew Flaschen
I'm not sure if the Wikipedia cookie is being treated as authentication for the purposes of this definition, but if it is, caching the site-wide CSS or JS seems unlikely to hurt (since it really is "public") - but obviously caching the user-specific CSS or JS would be bad.
Why? User-specific CSS and JS is still public.
That's true - I had assumed it was private (not that I've ever put anything there that I particularly cared about being private) - but you're correct, it's not ... which in turn raises another question:
------------------------------------------------------------------- root@bling:/var/www/hosts/mediawiki/wiki/skins# curl --silent --include --head "http://en.wikipedia.org/w/index.php?title=User:Nickj/monobook.js&action=..." HTTP/1.0 200 OK Date: Wed, 07 Feb 2007 06:17:28 GMT Server: Apache X-Powered-By: PHP/5.1.2 Cache-Control: private, s-maxage=0, max-age=2678400 Last-Modified: Wed, 31 Jan 2007 05:56:29 GMT Content-Type: text/javascript; charset=utf-8 X-Cache: MISS from sq16.wikimedia.org X-Cache-Lookup: MISS from sq16.wikimedia.org:80 Via: 1.0 sq16.wikimedia.org:80 (squid/2.6.STABLE9) Connection: close
root@bling:/var/www/hosts/mediawiki/wiki/skins# curl --silent --include --head "http://en.wikipedia.org/w/index.php?title=User:Nickj/monobook.js&action=..."
[ ... snip 3 more tests and responses from servers that's aren't sq16.wikimedia.org ... ]
root@bling:/var/www/hosts/mediawiki/wiki/skins# curl --silent --include --head "http://en.wikipedia.org/w/index.php?title=User:Nickj/monobook.js&action=..." HTTP/1.0 200 OK Date: Wed, 07 Feb 2007 06:17:41 GMT Server: Apache X-Powered-By: PHP/5.1.2 Cache-Control: private, s-maxage=0, max-age=2678400 Last-Modified: Wed, 31 Jan 2007 05:56:29 GMT Content-Type: text/javascript; charset=utf-8 X-Cache: MISS from sq16.wikimedia.org X-Cache-Lookup: MISS from sq16.wikimedia.org:80 Via: 1.0 sq16.wikimedia.org:80 (squid/2.6.STABLE9) Connection: close
-------------------------------------------------------------------
So, the question is: Why is the cache-control line for raw custom JS and CSS marked as "private" ?
From http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.1 , it defines the
"private" directive as: ------------------------------- private Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache. This allows an origin server to state that the specified parts of the response are intended for only one user and are not a valid response for requests by other users. A private (non-shared) cache MAY cache the response. -------------------------------
So, if the web page is public anyway, and people *do* use other people's custom CSS & JS (as I'm doing), why mark it as private when action=raw, and thus defeat shared caching?
All the best, Nick.
Nick Jenkins wrote:
I'm not sure if the Wikipedia cookie is being treated as authentication for the purposes of this definition, but if it is, caching the site-wide CSS or JS seems unlikely to hurt (since it really is "public") - but obviously caching the user-specific CSS or JS would be bad.
Why? User-specific CSS and JS is still public.
That's true - I had assumed it was private (not that I've ever put anything there that I particularly cared about being private) - but you're correct, it's not ... which in turn raises another question:
root@bling:/var/www/hosts/mediawiki/wiki/skins# curl --silent --include --head "http://en.wikipedia.org/w/index.php?title=User:Nickj/monobook.js&action=..." HTTP/1.0 200 OK Date: Wed, 07 Feb 2007 06:17:41 GMT Server: Apache X-Powered-By: PHP/5.1.2 Cache-Control: private, s-maxage=0, max-age=2678400 Last-Modified: Wed, 31 Jan 2007 05:56:29 GMT Content-Type: text/javascript; charset=utf-8 X-Cache: MISS from sq16.wikimedia.org X-Cache-Lookup: MISS from sq16.wikimedia.org:80 Via: 1.0 sq16.wikimedia.org:80 (squid/2.6.STABLE9) Connection: close
So, the question is: Why is the cache-control line for raw custom JS and CSS marked as "private" ?
From http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.1 , it defines the
"private" directive as:
private Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache. This allows an origin server to state that the specified parts of the response are intended for only one user and are not a valid response for requests by other users. A private (non-shared) cache MAY cache the response.
So, if the web page is public anyway, and people *do* use other people's custom CSS & JS (as I'm doing), why mark it as private when action=raw, and thus defeat shared caching?
I have no idea. I suspect with widely used scripts like popups that could significantly degrade performance.
P.S. To clarify, user css and js can't be *edited* by other users (admins can edit everything)
Matthew Flaschen
Brion Vibber wrote:
I took a quick peek at the sampled squid log and found that CSS and JS files together are eating a lot of bandwidth; together they make up about 20% of what's served:
https://wikitech.leuksman.com/view/Squid_bandwidth_breakdown
(May be inaccurate due to coding mistakes in my counter or weird dupe caching effects.)
It should be possible to serve these files compressed through Apache with mod_gzip set up, which should squish them by probably 2/3.
Is there a reason everything shouldn't be compressed?
Matthew Flaschen
On Wed, Feb 07, 2007 at 12:55:35AM -0500, Matthew Flaschen wrote:
Brion Vibber wrote:
It should be possible to serve these files compressed through Apache with mod_gzip set up, which should squish them by probably 2/3.
Is there a reason everything shouldn't be compressed?
MediaWiki already compresses all wiki pages. But static files are being served from apache directly and thus aren't compressed currently.
jens
On Tue, Feb 06, 2007 at 05:17:44PM -0800, Brion Vibber wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I took a quick peek at the sampled squid log and found that CSS and JS files together are eating a lot of bandwidth; together they make up about 20% of what's served:
https://wikitech.leuksman.com/view/Squid_bandwidth_breakdown
(May be inaccurate due to coding mistakes in my counter or weird dupe caching effects.)
It should be possible to serve these files compressed through Apache with mod_gzip set up, which should squish them by probably 2/3.
All projects have the same codebase, so that main.css is the same for de.wikipedia.org and en.wikipedia.org. Can we move the common CSS and JS to a common address? This would reduce the number of identical copies that we store in the squids - and of course it would reduce the load time for users accessing different projects.
Regards,
jens
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Jens Frank wrote:
All projects have the same codebase, so that main.css is the same for de.wikipedia.org and en.wikipedia.org. Can we move the common CSS and JS to a common address?
In theory.
1) Where? Upload cluster or something else? 2) How to set up compression for it?
- -- brion vibber (brion @ pobox.com / brion @ wikimedia.org)
On 2/6/07, Brion Vibber brion@pobox.com wrote:
I took a quick peek at the sampled squid log and found that CSS and JS files together are eating a lot of bandwidth; together they make up about 20% of what's served:
https://wikitech.leuksman.com/view/Squid_bandwidth_breakdown
(May be inaccurate due to coding mistakes in my counter or weird dupe caching effects.)
It should be possible to serve these files compressed through Apache with mod_gzip set up, which should squish them by probably 2/3.
Another option (which won't require more sysadminning or server CPU) is to compress them "manually" by stripping all whitespace and comments.
For example, looking at http://en.wikipedia.org/skins-1.5/monobook/main.css?55 it appears that comments and whitespace comprise more than 20% of the bytes in the file:
maincss.gsub(//*.*?*//m, '').gsub(/\s{2,}/, ' ').length / maincss.length.to_f
=> 0.79468590655243
You could likely do better with a real CSS parser.
"Evan Martin" evanm@google.com wrote in message news:9f43d19d0702071316k29d43f31y3a2cfe8a18d7859d@mail.gmail.com...
On 2/6/07, Brion Vibber brion@pobox.com
wrote:
I took a quick peek at the sampled squid log and found that CSS and JS files together are eating a lot of bandwidth; together they make up about 20% of what's served:
https://wikitech.leuksman.com/view/Squid_bandwidth_breakdown
(May be inaccurate due to coding mistakes in my counter or weird dupe caching effects.)
It should be possible to serve these files compressed through Apache with mod_gzip set up, which should squish them by probably 2/3.
Another option (which won't require more sysadminning or server CPU) is to compress them "manually" by stripping all whitespace and comments.
For example, looking at http://en.wikipedia.org/skins-1.5/monobook/main.css?55 it appears that comments and whitespace comprise more than 20% of the bytes in the file:
maincss.gsub(//*.*?*//m, '').gsub(/\s{2,}/, ' ').length /
maincss.length.to_f
=> 0.79468590655243
You could likely do better with a real CSS parser.
However, that will make debugging very hard indeed. Not sure how big a consideration that is, but I know that I have often looked into the MW stylesheets to see how particular stylings are acheived when making my own css rules.
- Mark Clements (HappyDog)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Moin,
On Wednesday 07 February 2007 22:38:48 Mark Clements wrote:
"Evan Martin" evanm@google.com wrote in message news:9f43d19d0702071316k29d43f31y3a2cfe8a18d7859d@mail.gmail.com...
On 2/6/07, Brion Vibber brion@pobox.com
wrote:
[snip]
For example, looking at http://en.wikipedia.org/skins-1.5/monobook/main.css?55 it appears that comments and whitespace comprise more than 20% of the
bytes in the file:
maincss.gsub(//*.*?*//m, '').gsub(/\s{2,}/, ' ').length /
maincss.length.to_f
=> 0.79468590655243
You could likely do better with a real CSS parser.
However, that will make debugging very hard indeed. Not sure how big a consideration that is, but I know that I have often looked into the MW stylesheets to see how particular stylings are acheived when making my own css rules.
But shouldn't be a CSS-tidy being able to parse the file and spit it out nicely formatted again?
best wishes,
tels
- -- Signed on Wed Feb 7 22:33:58 2007 with key 0x93B84C15. View my photo gallery: http://bloodgate.com/photos PGP key on http://bloodgate.com/tels.asc or per email.
"I have always wished that my computer would be as easy to use as my telephone. My wish has come true. I no longer know how to use my telephone." - Bjarne Stroustrup, inventor of the C++ programming language
"Tels" nospam-abuse@bloodgate.com wrote in message news:200702072234.51476@bloodgate.com...
On Wednesday 07 February 2007 22:38:48 Mark Clements wrote:
However, that will make debugging very hard indeed. Not sure how big a consideration that is, but I know that I have often looked into the MW stylesheets to see how particular stylings are acheived when making my own css rules.
But shouldn't be a CSS-tidy being able to parse the file and spit it out nicely formatted again?
That's not what was being suggested. The suggestion was to remove _all_ whitespace and all comments.
- Mark Clements (HappyDog)
However, that will make debugging very hard indeed. Not sure how big a consideration that is, but I know that I have often looked into the MW stylesheets to see how particular stylings are acheived when making my own css rules.
- Mark Clements (HappyDog)
Just from a quick check of http://en.wikipedia.org/skins-1.5/monobook/main.css?55, removing all the possible white space will reduce it in size from 30882 bytes to 23411 bytes, saving 7471 bytes (24%). That's a drastic change in size, and if done would save a ton of bandwidth (over time). If you're concerned about readability, perhaps two files are warranted, a "readable" file and a "usable" file, and the readable one could be linked from a comment at the top of the usable one.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Mets501 wrote:
However, that will make debugging very hard indeed. Not sure how big a consideration that is, but I know that I have often looked into the MW stylesheets to see how particular stylings are acheived when making my own css rules.
- Mark Clements (HappyDog)
Just from a quick check of http://en.wikipedia.org/skins-1.5/monobook/main.css?55, removing all the possible white space will reduce it in size from 30882 bytes to 23411 bytes, saving 7471 bytes (24%). That's a drastic change in size, and if done would save a ton of bandwidth (over time).
It's small compared to actual compression, though.
- -- brion vibber (brion @ pobox.com / brion @ wikimedia.org)
Just from a quick check of http://en.wikipedia.org/skins-1.5/monobook/main.css?55, removing all the possible white space will reduce it in size from 30882 bytes to 23411
bytes,
saving 7471 bytes (24%). That's a drastic change in size, and if done
would
save a ton of bandwidth (over time).
It's small compared to actual compression, though.
-- brion vibber (brion @ pobox.com / brion @ wikimedia.org)
Yes, but it may be a good temporary measure. --Mets501
On 2/7/07, Mets501 mets501wiki@gmail.com wrote:
Yes, but it may be a good temporary measure. --Mets501
We've survived this long...
Has anyone tried to figure out why it seems on average users load only slightly more than two pages for every time they load css?
Is really so much of our traffic "one click from a search engine and then depart" that it is shifting so much of our hits towards CSS loads? ... if so, it may be an indication that something is wrong with our service from the perspective of our readers.
(Perhaps they are so used to seeing spam with Wikipedia content that they just hit "back". :) )
We've survived this long...
Has anyone tried to figure out why it seems on average users load only slightly more than two pages for every time they load css?
Is really so much of our traffic "one click from a search engine and then depart" that it is shifting so much of our hits towards CSS loads? ... if so, it may be an indication that something is wrong with our service from the perspective of our readers.
(Perhaps they are so used to seeing spam with Wikipedia content that they just hit "back". :) )
I think most people just search google, and then encounter Wikipedia for one or two pages. I see my friends working in the library or other places doing research, and they search first on Google, click the first or second result (which is often Wikipedia), read that page, and then move on to other sites. --Mets501
On 07/02/07, Mets501 mets501wiki@gmail.com wrote:
Yes, but it may be a good temporary measure.
Well, no, not if it takes us more effort to integrate this whitespace stripping into the software than it does to configure things so that this stuff is compressed properly.
Rob Church
Yes, but it may be a good temporary measure.
Well, no, not if it takes us more effort to integrate this whitespace stripping into the software than it does to configure things so that this stuff is compressed properly.
Rob Church
I actually wasn't thinking of working it into the software, but rather just removing the white spaces in the main monobook css file (which is all most users load anyway). --Mets501
On 07/02/07, Mets501 mets501wiki@gmail.com wrote:
I actually wasn't thinking of working it into the software, but rather just removing the white spaces in the main monobook css file (which is all most users load anyway).
Well, from a developer's point of view, that's not actually acceptable, because we have to be able to maintain the damn file.
Rob Church
I actually wasn't thinking of working it into the software, but rather
just
removing the white spaces in the main monobook css file (which is all
most
users load anyway).
Well, from a developer's point of view, that's not actually acceptable, because we have to be able to maintain the damn file.
Rob Church
Listen, it was just a suggestion. If it's not acceptable, no problem. You're the programming professional, I'm not. --Mets501
I actually wasn't thinking of working it into the software, but rather just removing the white spaces in the main monobook css file (which is all most users load anyway).
... Yeah, and the rest! ;-)
-------------------------------- root@bling:~/tmp# curl --silent --include "http://en.wikipedia.org/wiki/Main_Page" | grep -i "css" <style type="text/css" media="screen,projection">/*<![CDATA[*/ @import "/skins-1.5/monobook/main.css?55"; /*]]>*/</style> <link rel="stylesheet" type="text/css" media="print" href="/skins-1.5/common/commonPrint.css?55" /> <link rel="stylesheet" type="text/css" media="handheld" href="/skins-1.5/monobook/handheld.css?55" /> <!--[if lt IE 5.5000]><style type="text/css">@import "/skins-1.5/monobook/IE50Fixes.css?55";</style><![endif]--> <!--[if IE 5.5000]><style type="text/css">@import "/skins-1.5/monobook/IE55Fixes.css?55";</style><![endif]--> <!--[if IE 6]><style type="text/css">@import "/skins-1.5/monobook/IE60Fixes.css?55";</style><![endif]--> <!--[if IE 7]><style type="text/css">@import "/skins-1.5/monobook/IE70Fixes.css?55";</style><![endif]--> <style type="text/css">/*<![CDATA[*/ @import "/w/index.php?title=MediaWiki:Common.css&usemsgcache=yes&action=raw&ctype=text/css&smaxage=2678400"; @import "/w/index.php?title=MediaWiki:Monobook.css&usemsgcache=yes&action=raw&ctype=text/css&smaxage=2678400"; @import "/w/index.php?title=-&action=raw&gen=css&maxage=2678400"; root@bling:~/tmp# --------------------------------
That's six CSS files for most desktop browsers, and seven if you're using IE.
All the best, Nick.
On 2/7/07, Nick Jenkins nickpj@gmail.com wrote:
That's six CSS files for most desktop browsers, and seven if you're using IE.
Actually just four and five, respectively. The media-specific stylesheets will probably not be loaded by clients that aren't using that medium.
That's six CSS files for most desktop browsers, and seven if you're using IE.
Actually just four and five, respectively. The media-specific stylesheets will probably not be loaded by clients that aren't using that medium.
I can only speak to what Firefox and IE are actually doing.
My proxy logs for doing a shift-F5 refresh on a local test wiki install include the following get/response headers for Firefox:
======================================================== +++GET 1595+++ GET /wiki/skins/monobook/main.css?55 HTTP/1.1 Host: 192.168.0.64 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 Accept: text/css,*/*;q=0.1 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Referer: http://192.168.0.64/wiki/index.php/Test Cookie: wikidbUserName=Nickj; wikidb_session=178df0fe68c75834643af65dec9ec98a; wikidbLoggedOut=20070208020154 Pragma: no-cache Cache-Control: no-cache Connection: keep-alive Browser reload detected...
+++RESP 1595+++ HTTP/1.1 200 OK Date: Thu, 08 Feb 2007 05:02:41 GMT Server: Apache Last-Modified: Mon, 05 Feb 2007 23:57:09 GMT ETag: "709d-428c37389ff40" Accept-Ranges: bytes Vary: Accept-Encoding,User-Agent Content-Encoding: gzip Cache-Control: max-age=2592000, public Content-Length: 7763 Content-Type: text/css +++CLOSE 1595+++ BlockList 1596: in Bypass, line 69
+++GET 1596+++ GET /wiki/skins/common/commonPrint.css?55 HTTP/1.1 Host: 192.168.0.64 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 Accept: text/css,*/*;q=0.1 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Referer: http://192.168.0.64/wiki/index.php/Test Cookie: wikidbUserName=Nickj; wikidb_session=178df0fe68c75834643af65dec9ec98a; wikidbLoggedOut=20070208020154 Pragma: no-cache Cache-Control: no-cache Connection: keep-alive Browser reload detected...
+++RESP 1596+++ HTTP/1.1 200 OK Date: Thu, 08 Feb 2007 05:02:41 GMT Server: Apache Last-Modified: Mon, 05 Feb 2007 23:57:09 GMT ETag: "1479-428c37389ff40" Accept-Ranges: bytes Vary: Accept-Encoding,User-Agent Content-Encoding: gzip Cache-Control: max-age=2592000, public Content-Length: 1731 Content-Type: text/css +++CLOSE 1596+++ BlockList 1597: in Bypass, line 69
+++GET 1597+++ GET /wiki/skins/monobook/handheld.css?55 HTTP/1.1 Host: 192.168.0.64 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 Accept: text/css,*/*;q=0.1 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Referer: http://192.168.0.64/wiki/index.php/Test Cookie: wikidbUserName=Nickj; wikidb_session=178df0fe68c75834643af65dec9ec98a; wikidbLoggedOut=20070208020154 Pragma: no-cache Cache-Control: no-cache Connection: keep-alive Browser reload detected...
+++RESP 1597+++ HTTP/1.1 200 OK Date: Thu, 08 Feb 2007 05:02:41 GMT Server: Apache Last-Modified: Mon, 05 Feb 2007 23:57:09 GMT ETag: "5bc3-428c37389ff40" Accept-Ranges: bytes Vary: Accept-Encoding,User-Agent Content-Encoding: gzip Cache-Control: max-age=2592000, public Content-Length: 6631 Content-Type: text/css +++CLOSE 1597+++ BlockList 1598: in Bypass, line 69 ========================================================
... which is also reflected in the apache logs:
======================================================== 192.168.0.2 - - [08/Feb/2007:16:20:38 +1100] "GET /wiki/skins/monobook/main.css?55 HTTP/1.1" 200 7763 "http://192.168.0.64/wiki/index.php/Test" "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1" 192.168.0.2 - - [08/Feb/2007:16:20:39 +1100] "GET /wiki/skins/common/commonPrint.css?55 HTTP/1.1" 200 1731 "http://192.168.0.64/wiki/index.php/Test" "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1" 192.168.0.2 - - [08/Feb/2007:16:20:39 +1100] "GET /wiki/skins/monobook/handheld.css?55 HTTP/1.1" 200 6631 "http://192.168.0.64/wiki/index.php/Test" "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1" ========================================================
Similarly for IE 5.01:
======================================================== +++GET 1610+++ GET /wiki/skins/common/commonPrint.css?55 HTTP/1.1 Accept: */* Referer: http://192.168.0.64/wiki/index.php/Test Accept-Language: en-au Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; .NET CLR 1.1.4322) Host: 192.168.0.64 Pragma: no-cache Connection: keep-alive
+++GET 1611+++ GET /wiki/skins/monobook/main.css?55 HTTP/1.1 Accept: */* Referer: http://192.168.0.64/wiki/index.php/Test Accept-Language: en-au Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; .NET CLR 1.1.4322) Host: 192.168.0.64 Pragma: no-cache Connection: keep-alive BlockList 1612: in Bypass, line 69 Browser reload detected...
+++GET 1612+++ GET /wiki/skins/monobook/handheld.css?55 HTTP/1.1 Accept: */* Referer: http://192.168.0.64/wiki/index.php/Test Accept-Language: en-au Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; .NET CLR 1.1.4322) Host: 192.168.0.64 Pragma: no-cache Connection: keep-alive ========================================================
... which is also reflected in the apache logs:
======================================================== 192.168.0.2 - - [08/Feb/2007:16:21:53 +1100] "GET /wiki/skins/common/commonPrint.css?55 HTTP/1.1" 200 1731 "http://192.168.0.64/wiki/index.php/Main_Page" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; .NET CLR 1.1.4322)" 192.168.0.2 - - [08/Feb/2007:16:21:53 +1100] "GET /wiki/skins/monobook/handheld.css?55 HTTP/1.1" 200 6631 "http://192.168.0.64/wiki/index.php/Main_Page" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; .NET CLR 1.1.4322)" 192.168.0.2 - - [08/Feb/2007:16:21:53 +1100] "GET /wiki/skins/common/IEFixes.js?55 HTTP/1.1" 200 1344 "http://192.168.0.64/wiki/index.php/Main_Page" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; .NET CLR 1.1.4322)" 192.168.0.2 - - [08/Feb/2007:16:21:53 +1100] "GET /wiki/skins/monobook/main.css?55 HTTP/1.1" 200 7763 "http://192.168.0.64/wiki/index.php/Main_Page" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; .NET CLR 1.1.4322)" ========================================================
I.e. both browsers fetch all 3 media variant stylesheets, at least on a full refresh.
... and exactly why both browsers feel the need to load a media="handheld" stylesheet whilst running on a distinctly non-handheld device is a question that's probably best directed at the authors of those browsers ;-)
All the best, Nick.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Moin,
On Thursday 08 February 2007 06:45:07 Nick Jenkins wrote:
That's six CSS files for most desktop browsers, and seven if you're using IE.
Actually just four and five, respectively. The media-specific stylesheets will probably not be loaded by clients that aren't using that medium.
I can only speak to what Firefox and IE are actually doing.
[snip]
I.e. both browsers fetch all 3 media variant stylesheets, at least on a full refresh.
... and exactly why both browsers feel the need to load a media="handheld" stylesheet whilst running on a distinctly non-handheld device is a question that's probably best directed at the authors of those browsers ;-)
Have you filed bug reports with them? :)
best wishes,
tels
- -- Signed on Thu Feb 8 19:22:58 2007 with key 0x93B84C15. View my photo gallery: http://bloodgate.com/photos PGP key on http://bloodgate.com/tels.asc or per email.
"HOT PACKET ON SERVER ACTION! Click here for FREE ACCESS to streaming video of dirty packets penetrating badly-configured firewalls!!!" aanand (705284) on 2004-03-20 on /. about the Adult Bit and the Evil bit
That's six CSS files for most desktop browsers, and seven if you're using IE.
Actually just four and five, respectively. The media-specific stylesheets will probably not be loaded by clients that aren't using that medium.
I can only speak to what Firefox and IE are actually doing.
[snip]
I.e. both browsers fetch all 3 media variant stylesheets, at least on a full refresh.
... and exactly why both browsers feel the need to load a media="handheld" stylesheet whilst running on a distinctly non-handheld device is a question that's probably best directed at the authors of those browsers ;-)
Have you filed bug reports with them? :)
For Firefox: See: "Bug 79021 - All stylesheets are loaded regardless of rendering media" https://bugzilla.mozilla.org/show_bug.cgi?id=79021 [Open 5.5 years, no activity in past 3.5 years]
For IE: Don't care - reporting bugs to MS is an exercise in pain and frustration, involving having to provide your credit card details to do their QA for them on their products (and then they say "sorry, we can't help" - completely useless), or if you use the web feedback forms they just ignore you for 9 years (at which point they discontinue the product) even though it was clearly a 1-line trivial fix ... or at least these have been *my* experiences reporting bugs to MS. And as the piece-de-resistance, their IE bug database is now closed ( https://connect.microsoft.com/IE ) to both reads and writes. Nope - I'm leaving this one to a masochist.
All the best, Nick.
On 2/8/07, Nick Jenkins nickpj@gmail.com wrote:
For Firefox: See: "Bug 79021 - All stylesheets are loaded regardless of rendering media" https://bugzilla.mozilla.org/show_bug.cgi?id=79021 [Open 5.5 years, no activity in past 3.5 years]
For IE: Don't care - reporting bugs to MS is an exercise in pain and frustration, involving having to provide your credit card details to do their QA for them on their products (and then they say "sorry, we can't help" - completely useless), or if you use the web feedback forms they just ignore you for 9 years (at which point they discontinue the product) even though it was clearly a 1-line trivial fix ... or at least these have been *my* experiences reporting bugs to MS. And as the piece-de-resistance, their IE bug database is now closed ( https://connect.microsoft.com/IE ) to both reads and writes. Nope - I'm leaving this one to a masochist.
At least this explains why we have half as many CSS loads as page views. :)
At least this explains why we have half as many CSS loads as page views. :)
Actually, I think 20% is an *underestimate* of the CSS & JS loads. The way it's being measured is like this: ======================================================= + ("upload", re.compile(r"^http://upload.wikimedia.org/")), + ("article", re.compile(r"^http://%5B%5E/%5D+/wiki/")), + ("wikifiles", re.compile(r"^http://%5B%5E/%5D+/w/")), + ("skins-css", re.compile(r"^http://%5B%5E/%5D+/skins-1.5/.*%5C.css")), + ("skins-js", re.compile(r"^http://%5B%5E/%5D+/skins-1.5/.*%5C.js")), + ("skins-image", re.compile(r"^http://%5B%5E/%5D+/skins-1.5/.*%5C.(?:png%7Cgif%7Cjpg)"))] =======================================================
Now, for me, 4 of the 7 CSS stylesheets I'm loading would be counted as "wikifiles" : ======================================================= @import "/w/index.php?title=MediaWiki:Common.css&usemsgcache=yes&action=raw&ctype=text/css&smaxage=2678400"; @import "/w/index.php?title=MediaWiki:Monobook.css&usemsgcache=yes&action=raw&ctype=text/css&smaxage=2678400"; @import "/w/index.php?title=-&action=raw&gen=css&maxage=2678400&smaxage=0&ts=20070207033933"; @import "/w/index.php?title=User:Nickj/monobook.css&action=raw&ctype=text/css"; ======================================================= (The reason it's 7 rather than 6 stylesheets is that I have a custom stylesheet; in IE it'd be 8 stylesheets)
An extra rule for "css-wikifiles" that matched on "/w/ + "action=raw&ctype=text/css" or "action=raw&gen=css" could be useful. I recognise and acknowledge though that the original figures were just a very rough quick back-of-the-envelope thing, and any figures are good and welcome :-)
My very very crude guess though is that *maybe* 2/3rds of the "wikifiles" category is CSS or JS files - which would make the total bandwidth usage around 26% for CSS and JS.
And what's more, that figure would certainly be higher, if it were not for the dynamic content compression already in place for those files (but not for the skins/ files).
All the best, Nick.
Nick Jenkins wrote:
At least this explains why we have half as many CSS loads as page views. :)
Actually, I think 20% is an *underestimate* of the CSS & JS loads. The way it's being measured is like this:
- ("upload", re.compile(r"^http://upload.wikimedia.org/")),
- ("article", re.compile(r"^http://%5B%5E/%5D+/wiki/")),
- ("wikifiles", re.compile(r"^http://%5B%5E/%5D+/w/")),
- ("skins-css", re.compile(r"^http://%5B%5E/%5D+/skins-1.5/.*%5C.css")),
- ("skins-js", re.compile(r"^http://%5B%5E/%5D+/skins-1.5/.*%5C.js")),
- ("skins-image", re.compile(r"^http://%5B%5E/%5D+/skins-1.5/.*%5C.(?:png%7Cgif%7Cjpg)"))]
=======================================================
The MIME type in the response is logged, to me that would be the obvious way to distinguish between CSS and HTML.
-- Tim Starling
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Nick Jenkins wrote:
At least this explains why we have half as many CSS loads as page views. :)
Actually, I think 20% is an *underestimate* of the CSS & JS loads. The way it's being measured is like this:
- ("upload", re.compile(r"^http://upload.wikimedia.org/")),
- ("article", re.compile(r"^http://%5B%5E/%5D+/wiki/")),
- ("wikifiles", re.compile(r"^http://%5B%5E/%5D+/w/")),
- ("skins-css", re.compile(r"^http://%5B%5E/%5D+/skins-1.5/.*%5C.css")),
- ("skins-js", re.compile(r"^http://%5B%5E/%5D+/skins-1.5/.*%5C.js")),
- ("skins-image", re.compile(r"^http://%5B%5E/%5D+/skins-1.5/.*%5C.(?:png%7Cgif%7Cjpg)"))]
=======================================================
Now, for me, 4 of the 7 CSS stylesheets I'm loading would be counted as "wikifiles" :
Irrelevant, since I was measuring the CSS and JS *files* specifically.
Those put out through the wiki are already compressed.
- -- brion vibber (brion @ pobox.com / brion @ wikimedia.org)
On 2/8/07, Nick Jenkins nickpj@gmail.com wrote:
For Firefox: See: "Bug 79021 - All stylesheets are loaded regardless of rendering media" https://bugzilla.mozilla.org/show_bug.cgi?id=79021 [Open 5.5 years, no activity in past 3.5 years]
Well, of course, if you don't vote for it. :)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Simetrical wrote:
On 2/8/07, Nick Jenkins nickpj@gmail.com wrote:
For Firefox: See: "Bug 79021 - All stylesheets are loaded regardless of rendering media" https://bugzilla.mozilla.org/show_bug.cgi?id=79021 [Open 5.5 years, no activity in past 3.5 years]
Well, of course, if you don't vote for it. :)
Bah, voting is useless. It looks like fixing that bug will be tough design-wise, so cash would be better.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Moin,
On Friday 09 February 2007 00:25:19 Nick Jenkins wrote:
That's six CSS files for most desktop browsers, and seven if you're using IE.
Actually just four and five, respectively. The media-specific stylesheets will probably not be loaded by clients that aren't using that medium.
I can only speak to what Firefox and IE are actually doing.
[snip]
I.e. both browsers fetch all 3 media variant stylesheets, at least on a full refresh.
... and exactly why both browsers feel the need to load a media="handheld" stylesheet whilst running on a distinctly non-handheld device is a question that's probably best directed at the authors of those browsers ;-)
Have you filed bug reports with them? :)
For Firefox: See: "Bug 79021 - All stylesheets are loaded regardless of rendering media" https://bugzilla.mozilla.org/show_bug.cgi?id=79021 [Open 5.5 years, no activity in past 3.5 years]
I didn't want to sound rude, just making sure it's actually filed as bug :) Maybe someone should reply to this bug report along the lines of "this wastes a lot of bandwidth on sites like wikipedia"
For IE: Don't care - reporting bugs to MS is an exercise in pain and frustration, involving having to provide your credit card details to do their QA for them on their products (and then they say "sorry, we can't help" - completely useless), or if you use the web feedback forms they just ignore you for 9 years (at which point they discontinue the product) even though it was clearly a 1-line trivial fix ... or at least these have been *my* experiences reporting bugs to MS. And as the piece-de-resistance, their IE bug database is now closed ( https://connect.microsoft.com/IE ) to both reads and writes. Nope - I'm leaving this one to a masochist.
I completely understand. You might as well talk to the trees. At least the wind makes them sway understandingly.
Just for a laugh I tried the page you gave, and it redirects you to:
http://www.microsoft.com/windows/products/winfamily/ie/iefaq.mspx
And when you click there on "Internet Explorer Support", it goes to:
http://www.microsoft.com/windows/products/winfamily/ie/tab2
Which can not be found. I rest my case.
Best wishes,
Tels
- -- Signed on Fri Feb 9 10:13:45 2007 with key 0x93B84C15. View my photo gallery: http://bloodgate.com/photos PGP key on http://bloodgate.com/tels.asc or per email.
"Look, I'm about to buy me a double barreled sawed off shotgun and show Linus what I think about backspace and delete not working." -- some anonymous .signature
On 2/9/07, Tels nospam-abuse@bloodgate.com wrote:
Maybe someone should reply to this bug report along the lines of "this wastes a lot of bandwidth on sites like wikipedia"
No! Bad Tels! No bugspam!
On 2/9/07, Phil Boswell phil.boswell@gmail.com wrote:
Is there a particular reason why the "IE Fixes" files are loaded separately? Would it be feasible to conflate them into a single ifle, with the selectors for particular versions *inside* that file, and then simply load that one file for IE users?
No, because the IE comment conditionals only work in HTML comments, not CSS comments. (Ref: http://www.quirksmode.org/css/condcom.html.)
Nick Jenkins wrote:
That's six CSS files for most desktop browsers, and seven if you're using IE.
Is there a particular reason why the "IE Fixes" files are loaded separately? Would it be feasible to conflate them into a single ifle, with the selectors for particular versions *inside* that file, and then simply load that one file for IE users?
If I've understood the rest of this thread, the combined file could be compressed quite a lot, so the question would be whether loading one bigger file is more efficient than possibly loading 4 smaller files.
HTH HAND
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Phil Boswell wrote:
Nick Jenkins wrote:
That's six CSS files for most desktop browsers, and seven if you're using IE.
Is there a particular reason why the "IE Fixes" files are loaded separately? Would it be feasible to conflate them into a single ifle, with the selectors for particular versions *inside* that file, and then simply load that one file for IE users?
a) there isn't a CSS selector for "MSIE versions 5.5 and above but below 6.0" etc, whereas there are conditional comments in MSIE's HTML parser
b) there's no reason to make everyone else load four or five MSIE-version-specific style sets
c) it's easier to maintain with them separate
- -- brion vibber (brion @ pobox.com / brion @ wikimedia.org)
Just from a quick check of http://en.wikipedia.org/skins-1.5/monobook/main.css?55, removing all the possible white space will reduce it in size from 30882 bytes to 23411 bytes, saving 7471 bytes (24%). That's a drastic change in size, and if done would save a ton of bandwidth (over time).
It's small compared to actual compression, though.
Exactly. To quantify with mod_deflate :
------------------------------------ root@bling:/var/www/hosts/mediawiki/wiki# curl --silent --include --head http://192.168.0.64/wiki/skins/monobook/main.css?55 HTTP/1.1 200 OK Date: Thu, 08 Feb 2007 01:20:04 GMT Server: Apache Last-Modified: Mon, 05 Feb 2007 23:57:09 GMT ETag: "709d-428c37389ff40" Accept-Ranges: bytes Content-Length: 28829 Vary: Accept-Encoding,User-Agent Cache-Control: max-age=2592000, public Content-Type: text/css
root@bling:/var/www/hosts/mediawiki/wiki# curl --silent --include --head --header "Accept-Encoding: gzip,deflate" http://192.168.0.64/wiki/skins/monobook/main.css?55 HTTP/1.1 200 OK Date: Thu, 08 Feb 2007 01:20:12 GMT Server: Apache Last-Modified: Mon, 05 Feb 2007 23:57:09 GMT ETag: "709d-428c37389ff40" Accept-Ranges: bytes Vary: Accept-Encoding,User-Agent Content-Encoding: gzip Cache-Control: max-age=2592000, public Content-Length: 7763 Content-Type: text/css
root@bling:/var/www/hosts/mediawiki/wiki# ------------------------------------
I.e. 28829 bytes reduced to 7763 bytes saves 21066 bytes, which means a saving of 73%.
And you don't have shadow versions of files, and the version that gets sent to clients is still legible (and it's the same one that developers know and are familiar with).
[P.s. please ignore the "public" directive in the cache-control line above, it's not there by default, I've just manually overridden it to test stuff. ]
All the best, Nick.
-----Original Message----- From: wikitech-l-bounces@lists.wikimedia.org [mailto:wikitech-l-bounces@lists.wikimedia.org] On Behalf Of Mets501 Sent: 07 February 2007 21:50 To: 'Wikimedia developers' Subject: Re: [Wikitech-l] Bandwidth usage
However, that will make debugging very hard indeed. Not
sure how big a
consideration that is, but I know that I have often looked
into the MW
stylesheets to see how particular stylings are acheived
when making my own
css rules.
- Mark Clements (HappyDog)
Just from a quick check of http://en.wikipedia.org/skins-1.5/monobook/main.css?55, removing all the possible white space will reduce it in size from 30882 bytes to 23411 bytes, saving 7471 bytes (24%). That's a drastic change in size, and if done would save a ton of bandwidth (over time). If you're concerned about readability, perhaps two files are warranted, a "readable" file and a "usable" file, and the readable one could be linked from a comment at the top of the usable one.
Waste of time if your gzipping. 7,772 bytes for main.css@55.gz, yet a whole load of hassle if want to try and read it.
Jared
wikitech-l@lists.wikimedia.org