Steve Sanbeg wrote:
After the last batch of problems were fixed, enwiki was getting dumped almost weekly. But there was no dump in December, and since it seems to be held up dumping the full history with an ETA of Jan 22nd, it seems like it will be at least a few more weeks.
So I wanted to check on what kind of schedule to expect. Is this a temporary issue, so we can go back to frequent dumps again once this is done? Or were the frequent dumps a temporary thing, and this is the norm?
Would it be difficult to run the current version & full history concurrently on a separate schedule, to allow the current versions to be dumped more frequently?
Within the last year we've tried to schedule monthly releases at the very least. Most consumers are perfectly happy with that data set as the time required to go through the copy and process is long enough to where anything more recent is unnecessary.
Now with several fixes and the re compression of the text external storage we had gotten down to two week cycles for most if not all wikis.
That short release cycle was certainly attractive but not a hard requirement.
Since we've crept just over the month cycle I'm certainly tempted to run another non full history snapshot in order to catch up. Assuming its not too resource intensive.
--tomasz