RE: [Wikitech-l] Re: Yahoo! XML feed
by Constans, Camille (C.C.)
>
>
>On Apr 2, 2004, at 00:53, Jimmy Wales wrote:
>> It shouldn't run more than once per day at first. I'm not sure what
>> their goals are with respect to how often they would *like*
>to receive
>> it, but daily is a fine start.
>
>It would take hours just to run a complete dump, which would be the
>equivalent of a sizeable fraction of our total daily page views. (Best
>case might be 100ms per page for 240,000 pages =~ 6 hours 40 minutes)
>
>If we're going to run something like this daily, some sort of
>incremental updates are a must, though we can probably get away with
>stuffing the saved data per page in a database or such and slurping it
>back out fairly quickly.
>
>-- brion vibber (brion @ pobox.com)
Incremental need to check in the database each time if this is the last version, no ?
Why not using a slave db server on less loaded server ? (like zwinger for example (need to add another ide drive))
Shaihulud