On 10/01/11 22:13, Ariel T. Glenn wrote:
So "soon" took longer than I would have
liked. However, we are up and
running with the new code. I have started a few processes going and
over the next few days I will ramp it up to the usual number. In
particular I want to start a separate job for the larger wikis so that
the smaller jobs don't get trapped behind them.
Guess I'd better go update the various pages on wikitech now.
Ariel
Thanks Ariel, that's good to hear.
Would it be possible to take this a step further, and for a single job
to be started up just for enwiki?
enwiki is unique among all the dumps in that it is the only one that
regularly fails more often than it succeeds; even partial dumps are
better than none, and enwiki also takes longer than any other dump
before it (typically) fails, so retrying it more aggressively than
others -- and independently of them, so it does not hold the other wikis
up -- would seem appropriate.
Thus, under this proposal, there would be three jobs running:
* enwiki
* other large wikis
* all small wikis
-- Neil