Hi all, After the previous thread, I ask out of curiosity, what if there were just a small number of servers spread around the world, not owned by the WMF. Something like:
- Users can access en.wikipedia.org, en.wikipedia-brazil.org, en.wikipedia-japan.org etc interchangeably - Each site is a complete copy of the database, and capable of serving independently - Every change on one database is immediately replicated to the others - To make this worthwhile, all the servers except the original are owned and operated by third parties on a sponsorship arrangement, presumably meaning they get to discreetly stick a logo somewhere
Is this sort of thing technically feasible? The clear advantage is faster response time for people near one of the overseas hosts, especially when just browsing. I can see obvious problems with the replication (ie, two competing requests arriving simultaneously), and what happens when a link fails for an extended period of time. Perhaps a simpler model:
- Read requests are fulfilled by these distributed servers - All write requests are sent to the one central server which immediately pushes out a "dirty page" notification (but not page content) to the other servers - The distributed servers fetch updated pages when a user requests a "dirty page", or perhaps after some time period, to avoid the whole database becoming too out of date
This model would rely on the fact that the vast majority of requests are reads, not writes, and attempts to reduce the impact of a page which is heavily modified by one server, while infrequently requested on another.
Thoughts? I'm obviously not a network or database engineer so I'm just wondering if this would be a workable or useful solution. It at least avoids the problems of untrusted servers, unreliable servers, and gives an additional benefit in responsiveness.
Steve
Steve Bennett wrote:
Hi all, After the previous thread, I ask out of curiosity, what if there were just a small number of servers spread around the world, not owned by the WMF. Something like:
- Users can access en.wikipedia.org, en.wikipedia-brazil.org,
en.wikipedia-japan.org etc interchangeably
- Each site is a complete copy of the database, and capable of serving
independently
- Every change on one database is immediately replicated to the others
- To make this worthwhile, all the servers except the original are
owned and operated by third parties on a sponsorship arrangement, presumably meaning they get to discreetly stick a logo somewhere
[...]
As far as I know, you've just described (more or less) the MySQL Cluster database: http://mysql.com/products/database/cluster/
Boris
Steve Bennett wrote:
Hi all, After the previous thread, I ask out of curiosity, what if there were just a small number of servers spread around the world, not owned by the WMF. Something like:
- Users can access en.wikipedia.org, en.wikipedia-brazil.org,
en.wikipedia-japan.org etc interchangeably
- Each site is a complete copy of the database, and capable of serving
independently
- Every change on one database is immediately replicated to the others
- To make this worthwhile, all the servers except the original are
owned and operated by third parties on a sponsorship arrangement, presumably meaning they get to discreetly stick a logo somewhere
Is this sort of thing technically feasible? The clear advantage is faster response time for people near one of the overseas hosts, especially when just browsing.
We can acheive faster response time just by having HTTP proxies with keep-alive working properly. Response times of RTT to Florida plus server overhead should be possible for most of Europe and Asia, currently it's more like 6 x RTT + overhead, due to TCP connection setup and slow-start.
The great advantage to this is that it can and will be done in the next few days, we don't have to wait for all these pipe dreams to magically come to fruition.
-- Tim Starling
On 2/19/07, Tim Starling tstarling@wikimedia.org wrote:
The great advantage to this is that it can and will be done in the next few days, we don't have to wait for all these pipe dreams to magically come to fruition.
Yeah, I like that advantage :)
Steve
On 19/02/07, Tim Starling tstarling@wikimedia.org wrote:
The great advantage to this is that it can and will be done in the next few days, we don't have to wait for all these pipe dreams to magically come to fruition.
Good, solid solution that works. Superb!
Rob Church
wikitech-l@lists.wikimedia.org