So this increases the frequency of dumps for small wikis, great.
But this means that the time beetween two dumps of the big wikis is
_at_least_ the sum of the times needed to dump each one of the big
wikis... more than 10, 12 weeks, not counting any failure ? I don't
think that you really want to do this
2008/10/12 Robert Ullmann <rlullmann(a)gmail.com>om>:
On Sat, Oct 11, 2008 at 6:30 PM, Chad
<innocentkiller(a)gmail.com> wrote:
On Sat, Oct 11, 2008 at 10:26 AM, Thomas Dalton <thomas.dalton(a)gmail.com>wrote;wrote:
Indeed, you can make some dumps more frequent at
the expense of making
others less frequent. No-one has yet explained why small dumps should
be more frequent that large ones.
Or, for that matter, what is gained by more frequent dumps, period.
6 weeks isn't a massive amount of time...
on en,wikt, we have several dozen reports and such that need updating
to manage a lot of details. 6 weeks is *interminable*. Which is why we
are running daily incrementals now
Why should small dumps be more frequent than large ones? Because they
should be weekly. The problem is that the large ones take much too
long, and clog the queue.
This is not rocket science people, it just needs one thread that
doesn't get blocked. Simple to do. In the loop in findAndLockNextWiki
(for db), do
if '--small' in sys.argv[1:] and db.description() in ['enwiki',
'dewiki', 'frwiki', 'plwiki', 'jawiki, 'itwiki',
'nlwiki', 'ptwiki',
'eswiki', 'ruwiki']: continue
that is all. Then run one thread with --small.
(those are the 10 largest pedias, the ones with more than 10M edits)
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l