On Thu, Oct 9, 2008 at 9:25 PM, Brion Vibber brion@wikimedia.org wrote:
Should be pretty easy to set up a small-projects-only thread.
Bigger changes to the dumps generation should come in the next couple months to make it quicker and more reliable...
If we look at the process right now, there are two threads: one is doing enwiki, the other hewiki. The enwiki thread isn't even to pages-articles yet, and will run for weeks. The hewiki dump will complete in a day or so, but then next on deck is dewiki, which takes at least a week.
So with things running now, it will be a week or two before any other projects get anything.
As you say, it would be easy to make threads limited to smaller projects; I'd suggest adding an option (-small or something) that just has the code skip [en, de, zh, he ...]wiki when looking for the least-recently completed task. The code list should be 10-15 of the biggest 'pedias, and possibly commons. Then start two small threads, and everything should go well?
best, Robert