On Mon, 04 Jan 2010 18:11:33 -0800, Tomasz Finc wrote:
Steve Sanbeg wrote:
After the last batch of problems were fixed,
enwiki was getting dumped
almost weekly. But there was no dump in December, and since it seems to
be held up dumping the full history with an ETA of Jan 22nd, it seems
like it will be at least a few more weeks.
So I wanted to check on what kind of schedule to expect. Is this a
temporary issue, so we can go back to frequent dumps again once this is
done? Or were the frequent dumps a temporary thing, and this is the
norm?
Would it be difficult to run the current version & full history
concurrently on a separate schedule, to allow the current versions to be
dumped more frequently?
Within the last year we've tried to schedule monthly releases at the very
least. Most consumers are perfectly happy with that data set as the time
required to go through the copy and process is long enough to where
anything more recent is unnecessary.
Now with several fixes and the re compression of the text external storage
we had gotten down to two week cycles for most if not all wikis.
That short release cycle was certainly attractive but not a hard
requirement.
Since we've crept just over the month cycle I'm certainly tempted to run
another non full history snapshot in order to catch up. Assuming its
not too resource intensive.
--tomasz
OK, thanks. It would definitely be useful for me if we can keep two week
cycles most of the time, and not slip to more than a month. I was
concerned that getting more frequent data was dependent on something else
failing for the dump to be created more frequently, which would be a
shame. Anyway, I'll keep checking for the next dump.
Thanks
-Steve