On Thu, Oct 9, 2008 at 9:25 PM, Brion Vibber <brion(a)wikimedia.org> wrote:
Should be pretty easy to set up a small-projects-only thread.
Bigger changes to the dumps generation should come in the next couple
months to make it quicker and more reliable...
If we look at the process right now, there are two threads: one is doing
enwiki, the other hewiki. The enwiki thread isn't even to pages-articles
yet, and will run for weeks. The hewiki dump will complete in a day or so,
but then next on deck is dewiki, which takes at least a week.
So with things running now, it will be a week or two before any other
projects get anything.
As you say, it would be easy to make threads limited to smaller projects;
I'd suggest adding an option (-small or something) that just has the code
skip [en, de, zh, he ...]wiki when looking for the least-recently completed
task. The code list should be 10-15 of the biggest 'pedias, and possibly
commons. Then start two small threads, and everything should go well?
best, Robert