Hello,
just read about Wikipedia and the Wiktionary this morning in www.heise.de for the first time and I'm impressed ! Good work.
getting on-topic again, I could not find any information on setting up mirrors (the How to become a Wikipedia hacker page is a bit incomplete :) Is there any best-practice way to do it, which avoids transferring SQL Dumps of the whole database every day ?
Do MySQL or PostgreSQL have +working+ replication, transmitting just the changes/additions, has anybody ever used them to replicate Wikipedia ?
cheers, buraq
On Wed, Jan 22, 2003 at 11:35:43AM +0100, buraq wrote:
Hello,
just read about Wikipedia and the Wiktionary this morning in www.heise.de for the first time and I'm impressed ! Good work.
getting on-topic again, I could not find any information on setting up mirrors (the How to become a Wikipedia hacker page is a bit incomplete :) Is there any best-practice way to do it, which avoids transferring SQL Dumps of the whole database every day ?
Do MySQL or PostgreSQL have +working+ replication, transmitting just the changes/additions, has anybody ever used them to replicate Wikipedia ?
PostgreSQL does have working replication. It is scheduled to be implemented in mod_wiki. I've just started cutting code; things have matured to a point that I am now using a clone of Donald Knuths "CWEB" literate programming tools to write this. This should hopefully make it easier for other people to pick up the source code and make their changes to it, since it will be extensively documented.
I have also set up a Savannah project for mod_wiki, and once I have some minimal wiki functionality, I will do the first checkin, and we will be off to the races.
Jonathan
On mer, 2003-01-22 at 02:35, buraq wrote:
just read about Wikipedia and the Wiktionary this morning in www.heise.de for the first time and I'm impressed ! Good work.
Wilkommen aus Wikipedia!
getting on-topic again, I could not find any information on setting up mirrors (the How to become a Wikipedia hacker page is a bit incomplete :) Is there any best-practice way to do it, which avoids transferring SQL Dumps of the whole database every day ?
Do MySQL or PostgreSQL have +working+ replication, transmitting just the changes/additions, has anybody ever used them to replicate Wikipedia ?
MySQL replication _supposedly_ works... I'm considering setting it up on our own server so that slow read-only operations (like the SQL dumps) can be done on the local mirror without the main database locking and grinding to a complete halt for forty minutes. I haven't yet looked into the setup details in earnest though.
Allowing remote servers to sync that way should be possible. Can we leave out certain tables? Eg, the user table, which contains sensitive information -- password hashes and e-mail addresses -- that we'd rather not have zipping about the net willy-nilly.
-- brion vibber (brion @ pobox.com)
wikitech-l@lists.wikimedia.org