Hi,
today we came over 10k HTTP requests per second (even with inter-squid traffic eliminated). Especially thanks to Mark and Tim, who've been improving our caching, as well as doing lots of other work, and achieved incredible results (while I was slacking). Really, thanks!
Domas
On 05/01/06 13:53, Domas Mituzas wrote:
Hi,
today we came over 10k HTTP requests per second (even with inter-squid traffic eliminated). Especially thanks to Mark and Tim, who've been improving our caching, as well as doing lots of other work, and achieved incredible results (while I was slacking). Really, thanks!
Domas
Wow. That is a cool statistic. Way to go on tuning those systems. *10k/sec is a lot* Time to think about writing an article for the Linux Journal? Hmmm, "Performance Tuning at the Wikipedia"...
-- Enjoy, Jeff
Jeff Carr wrote:
today we came over 10k HTTP requests per second
Wow. That is a cool statistic. Way to go on tuning those systems. *10k/sec is a lot* Time to think about writing an article for the Linux Journal? Hmmm, "Performance Tuning at the Wikipedia"...
Yeah but it's old news, we're at 12k/s already... ;)
On May 1 2006, Domas Mituzas wrote:
Hi,
today we came over 10k HTTP requests per second (even with inter-squid traffic eliminated). Especially thanks to Mark and Tim, who've been improving our caching, as well as doing lots of other work, and achieved incredible results (while I was slacking). Really, thanks!
One and a half year ago we were proud of 10,000 HTTP requests/s, As of today we're doing over 50,000 requests/s during peak.
http://www.nedworks.org/~mark/reqstats/reqstats-daily.png http://www.nedworks.org/~mark/reqstats/reqstats-yearly.png
We're still growing strong. :)
Thanks again to the entire technical team and everyone else who made this possible on our small budget; without further software/architecture improvements and optimizations since last year, this would not have been possible. :)
Yes, a great commendation to the whole tech team.
You guys are doing great - keep up the good work!
Kind regards,
E English Wikipedia IRC: TheLetterE Email: e.wikipedia@gmail.com
-------------------------------------------------- From: "Mark Bergsma" mark@wikimedia.org Sent: Tuesday, November 27, 2007 6:10 AM To: "Wikimedia developers" wikitech-l@lists.wikimedia.org Subject: Re: [Wikitech-l] <s>10k</s> 50k!
On May 1 2006, Domas Mituzas wrote:
Hi,
today we came over 10k HTTP requests per second (even with inter-squid traffic eliminated). Especially thanks to Mark and Tim, who've been improving our caching, as well as doing lots of other work, and achieved incredible results (while I was slacking). Really, thanks!
One and a half year ago we were proud of 10,000 HTTP requests/s, As of today we're doing over 50,000 requests/s during peak.
http://www.nedworks.org/~mark/reqstats/reqstats-daily.png http://www.nedworks.org/~mark/reqstats/reqstats-yearly.png
We're still growing strong. :)
Thanks again to the entire technical team and everyone else who made this possible on our small budget; without further software/architecture improvements and optimizations since last year, this would not have been possible. :)
-- Mark Bergsma mark@wikimedia.org System & Network Administrator, Wikimedia Foundation
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
One and a half year ago we were proud of 10,000 HTTP requests/s, As of today we're doing over 50,000 requests/s during peak.
400% growth in 18 months? Wow! By my calculations, that's about 2% a *week*. Congratulations to everyone involved in preventing the servers from exploding, you've done an amazing job!
Mark Bergsma wrote:
http://www.nedworks.org/~mark/reqstats/reqstats-daily.png http://www.nedworks.org/~mark/reqstats/reqstats-yearly.png
Is the blue (pmtpa = Florida) a cost for WMF, and the green (knams = Amsterdam) a cost for someone else (Kennisnet + Wikimedia Deutschland)? How much could WMF save if there was a knams-like installation in, say, Chicago, New York or San Francisco, paid for by a Kennisnet-like sponsor?
When I donate to the WMF, do I help paying for the blue only, or also for the green part of these diagrams?
On Tue, Nov 27, 2007 at 09:10:38AM +0100, Lars Aronsson wrote:
Mark Bergsma wrote:
http://www.nedworks.org/~mark/reqstats/reqstats-daily.png http://www.nedworks.org/~mark/reqstats/reqstats-yearly.png
Is the blue (pmtpa = Florida) a cost for WMF, and the green (knams = Amsterdam) a cost for someone else (Kennisnet + Wikimedia Deutschland)?
The traffic in KNAMS is currently covered by Kennisnet. Kennisnet and WM.de provided the servers in KNAMS, WMF paid for manpower and some hardware, e.g. network switches and router interfaces. (AFAIK, others may be able to give a more detailed overview)
How much could WMF save if there was a knams-like installation in, say, Chicago, New York or San Francisco, paid for by a Kennisnet-like sponsor?
I don't have any figures about this.
When I donate to the WMF, do I help paying for the blue only, or also for the green part of these diagrams?
Donations to the WMF will be used to cover the blue part, parts of the green part, and overall costs of the project, which are not only IT related.
Regards,
jens
Hi!
When I donate to the WMF, do I help paying for the blue only, or also for the green part of these diagrams?
_lots_ of donations are to build the infrastructure for tiny black line at the bottom. That is the place where we handle various difficult part of the logic - editing, reparsing of huge articles, analysis, search, storage, etc.
BR,
On Nov 27, 2007 3:10 AM, Lars Aronsson lars@aronsson.se wrote:
Is the blue (pmtpa = Florida) a cost for WMF, and the green (knams = Amsterdam) a cost for someone else (Kennisnet + Wikimedia Deutschland)? How much could WMF save if there was a knams-like installation in, say, Chicago, New York or San Francisco, paid for by a Kennisnet-like sponsor?
When I donate to the WMF, do I help paying for the blue only, or also for the green part of these diagrams?
Wikimedia Deutschland doesn't support infrastructure in this manner. Their sponsorship at knams is the toolserver.
Space and connectivity at Knams is provided by Kennisnet. The equipment is provided by the WMF. Yahoo has provided equipment and connectivity in Korea for the Yaseo cluster. Traffic from Tampa is paid for by Wikimedia at impressively low rates.
Spreading traffic out to many locations can reduce Wikimedia's ability to lower costs by putting more traffic on smaller amounts of equipment and by negotiating lower rates for the traffic (like many other things Internet capacity costs much less per unit when bought in bulk). As a result the whole costs must be considered and weighed carefully. The increases in traffic actually make increased distribution more viable. It must also be kept in mind that when Wikimedia accepts enormous gifts of capacity it comes with the risk of dependence.
Fortunately, Mark has been building a long term plan for network capacity which will continue to lower the incremental cost for connectivity as Wikimedia's connectivity needs rise without significant risks of dependence. There will be more announcements related to this in the coming months.
Hoi, This is not completely correct. A lot of the systems at KNAMS have also been provided by Kennisnet. This was certainly true for the first batch of servers.
The cost of connectivity is different in countries in Africa and Asia. The pipes going in are so small that getting data from outside the country makes the quality drop substantially. For these countries a conventional cost argument is wrong.
Having some servers in countries like Kyrgyzstan will not really make a difference from a dependency point of view. Thanks, GerardM
On Nov 30, 2007 12:52 AM, Gregory Maxwell gmaxwell@gmail.com wrote:
On Nov 27, 2007 3:10 AM, Lars Aronsson lars@aronsson.se wrote:
Is the blue (pmtpa = Florida) a cost for WMF, and the green (knams = Amsterdam) a cost for someone else (Kennisnet + Wikimedia Deutschland)? How much could WMF save if there was a knams-like installation in, say, Chicago, New York or San Francisco, paid for by a Kennisnet-like sponsor?
When I donate to the WMF, do I help paying for the blue only, or also for the green part of these diagrams?
Wikimedia Deutschland doesn't support infrastructure in this manner. Their sponsorship at knams is the toolserver.
Space and connectivity at Knams is provided by Kennisnet. The equipment is provided by the WMF. Yahoo has provided equipment and connectivity in Korea for the Yaseo cluster. Traffic from Tampa is paid for by Wikimedia at impressively low rates.
Spreading traffic out to many locations can reduce Wikimedia's ability to lower costs by putting more traffic on smaller amounts of equipment and by negotiating lower rates for the traffic (like many other things Internet capacity costs much less per unit when bought in bulk). As a result the whole costs must be considered and weighed carefully. The increases in traffic actually make increased distribution more viable. It must also be kept in mind that when Wikimedia accepts enormous gifts of capacity it comes with the risk of dependence.
Fortunately, Mark has been building a long term plan for network capacity which will continue to lower the incremental cost for connectivity as Wikimedia's connectivity needs rise without significant risks of dependence. There will be more announcements related to this in the coming months.
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hi!
The cost of connectivity is different in countries in Africa and Asia. The pipes going in are so small that getting data from outside the country makes the quality drop substantially. For these countries a conventional cost argument is wrong.
In many of those countries end-user points are often connected by satellites, so no local caches would help. Unless, of course, we'd have our own satellite :)
Having some servers in countries like Kyrgyzstan will not really make a difference from a dependency point of view.
That will add yet another cluster to manage, what proved to be painful by french experiment. And amount of work required per page view would be 1000x bigger than for knams.
Of course, I'm talking like a lazy tech here, but still we can't allow luxury of reaching into every ISP in every developing country.
My $0.02
On Nov 30, 2007 12:35 PM, Thomas Dalton thomas.dalton@gmail.com wrote:
In many of those countries end-user points are often connected by satellites, so no local caches would help. Unless, of course, we'd have our own satellite :)
Think big, huh? ;)
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Actually, one would need three geostationary satellites to cover the whole earth. Then still you don't have coverage in Antartica and Northern Russia and Canada, which would mean that you need additional satellites in very eccentric polar orbits. Maybe an idea for fundraiser 2050.
Bryan
On Fri, Nov 30, 2007 at 01:02:17PM +0100, Bryan Tong Minh wrote:
On Nov 30, 2007 12:35 PM, Thomas Dalton thomas.dalton@gmail.com wrote:
In many of those countries end-user points are often connected by satellites, so no local caches would help. Unless, of course, we'd have our own satellite :)
Think big, huh? ;)
Actually, one would need three geostationary satellites to cover the whole earth. Then still you don't have coverage in Antartica and Northern Russia and Canada, which would mean that you need additional satellites in very eccentric polar orbits. Maybe an idea for fundraiser 2050.
Well...
NASA *did* just announce a cheaper satellite launch profile plan...
Cheers, -- jra
On 30/11/2007, Jay R. Ashworth jra@baylink.com wrote:
On Fri, Nov 30, 2007 at 01:02:17PM +0100, Bryan Tong Minh wrote:
On Nov 30, 2007 12:35 PM, Thomas Dalton thomas.dalton@gmail.com wrote:
In many of those countries end-user points are often connected by satellites, so no local caches would help. Unless, of course, we'd have our own satellite :)
Think big, huh? ;)
Actually, one would need three geostationary satellites to cover the whole earth. Then still you don't have coverage in Antartica and Northern Russia and Canada, which would mean that you need additional satellites in very eccentric polar orbits. Maybe an idea for fundraiser 2050.
Well...
NASA *did* just announce a cheaper satellite launch profile plan...
Or we could get every Wikipedian to build a high altitude blimp and anchor it to their house and use them as boosters to get Wikipedia to everyone. While it might be more expensive overall, it would allow us to get started sooner.
Thomas Dalton wrote:
In many of those countries end-user points are often connected by satellites, so no local caches would help. Unless, of course, we'd have our own satellite :)
Think big, huh? ;)
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Satellite distribution by IP multicast is perhaps not as daft as it sounds: the nature of satellite transmission means that it would cost the same to send a one-way update datastream to local caches on an entire continent as it would to send it to a single point on that continent.
As a result, only a tiny amount of IP capacity on existing satellites would need to be purchased ot keep local caches up-to-date. It still wouldn't solve the problem of point-to-point distribution in countries where no suitable point-to-point terrestrial infrastructure exists, but it might work well enough for local university caches.
-- Neil
Hi there,
Gregory Maxwell schrieb am 30.11.2007 00:52:
On Nov 27, 2007 3:10 AM, Lars Aronsson lars@aronsson.se wrote:
Is the blue (pmtpa = Florida) a cost for WMF, and the green (knams = Amsterdam) a cost for someone else (Kennisnet + Wikimedia Deutschland)? How much could WMF save if there was a knams-likeSquid
When I donate to the WMF, do I help paying for the blue only, or also for the green part of these diagrams?
Wikimedia Deutschland doesn't support infrastructure in this manner. Their sponsorship at knams is the toolserver.
Well, that is definitely not correct.
Just from my head, without checking further sources: While it is correct that all four servers of the toolserver (zedler, hemlock, yarrow, [vandale?]) are provided by Wikimedia Deutschland (two donated by SUN, two bought by WM-DE), this is only a small part of our work.
So for example Wikimedia Deutschland is also providing half of the Squid proxies in Amsterdam (15x servers/initial costs more than 60k euros) and the backup server (~14k euros). So it is more than appropriate to say that WM-DE is supporting the infrastructure.
Bye, Tim (WM-DE board member)
On 26/11/2007, Mark Bergsma mark@wikimedia.org wrote:
On May 1 2006, Domas Mituzas wrote:
today we came over 10k HTTP requests per second (even with inter-squid traffic eliminated). Especially thanks to Mark and Tim, who've been improving our caching, as well as doing lots of other work, and achieved incredible results (while I was slacking). Really, thanks!
One and a half year ago we were proud of 10,000 HTTP requests/s, As of today we're doing over 50,000 requests/s during peak. http://www.nedworks.org/~mark/reqstats/reqstats-daily.png http://www.nedworks.org/~mark/reqstats/reqstats-yearly.png We're still growing strong. :) Thanks again to the entire technical team and everyone else who made this possible on our small budget; without further software/architecture improvements and optimizations since last year, this would not have been possible. :)
Mark, could you please submit this (with links to whatever other technical stuff would interest the geeks) to the Why Give? blog? I think it answers the question marvellously. Perhaps with a last-12-months version of that 'yearly' graph.
http://meta.wikimedia.org/wiki/Fundraising_2007/Why_Give_blog
- d.
David Gerard wrote:
Mark, could you please submit this (with links to whatever other technical stuff would interest the geeks) to the Why Give? blog? I think it answers the question marvellously.
If web traffic increases, causing a higher bandwidth cost, that is then covered by Kennisnet, that *doesn't* answer the question why one should donate to the Wikimedia Foundation. (Perhaps one should donate to Kennisnet, or at least say thanks to all the taxpayers in the Netherlands.)
I'm not against donations, but I'm against blurring these facts. We should know, rather than guess, what is the best solution and who can pick up which bill.
As Domas pointed out, much cost is involved in the thin black line at the bottom of that diagram. The diagram doesn't show why one should need to donate. The amount of web traffic that is handled is indeed impressive. But it isn't necessarily related to costs.
Lars Aronsson wrote:
David Gerard wrote:
Mark, could you please submit this (with links to whatever other technical stuff would interest the geeks) to the Why Give? blog? I think it answers the question marvellously.
If web traffic increases, causing a higher bandwidth cost, that is then covered by Kennisnet, that *doesn't* answer the question why one should donate to the Wikimedia Foundation. (Perhaps one should donate to Kennisnet, or at least say thanks to all the taxpayers in the Netherlands.)
And the servers needed to serve that traffic are free? :) And the network equipment? And the increased traffic in Florida? And the people that manage those servers? etc.
I'm not against donations, but I'm against blurring these facts. We should know, rather than guess, what is the best solution and who can pick up which bill.
As Domas pointed out, much cost is involved in the thin black line at the bottom of that diagram. The diagram doesn't show why one should need to donate. The amount of web traffic that is handled is indeed impressive. But it isn't necessarily related to costs.
It's *very* related to costs.
wikitech-l@lists.wikimedia.org