On Sat, Oct 11, 2008 at 6:30 PM, Chad <innocentkiller(a)gmail.com> wrote:
On Sat, Oct 11, 2008 at 10:26 AM, Thomas Dalton <thomas.dalton(a)gmail.com>wrote;wrote:
Indeed, you can make some dumps more frequent at
the expense of making
others less frequent. No-one has yet explained why small dumps should
be more frequent that large ones.
Or, for that matter, what is gained by more frequent dumps, period.
6 weeks isn't a massive amount of time...
on en,wikt, we have several dozen reports and such that need updating
to manage a lot of details. 6 weeks is *interminable*. Which is why we
are running daily incrementals now
Why should small dumps be more frequent than large ones? Because they
should be weekly. The problem is that the large ones take much too
long, and clog the queue.
This is not rocket science people, it just needs one thread that
doesn't get blocked. Simple to do. In the loop in findAndLockNextWiki
(for db), do
if '--small' in sys.argv[1:] and db.description() in ['enwiki',
'dewiki', 'frwiki', 'plwiki', 'jawiki, 'itwiki',
'nlwiki', 'ptwiki',
'eswiki', 'ruwiki']: continue
that is all. Then run one thread with --small.
(those are the 10 largest pedias, the ones with more than 10M edits)