On 23/04/13 18:31, Ariel T. Glenn wrote:
The long version is that the pages-logging file is
already about half
the size of en wp's table, and that the number of edits per minute is
much larger, see:
https://wikipulse.herokuapp.com/
There's a lot of deletion and a lot of churn too due to the dispatch
mechanism.
Also, they apparently have RCPatrol enabled and a pile of bots, which
means that the log consists of 99% entries 'bot X editing Y marked it as
autopatrolled'.
These things in combo turn out to be the perfect storm for my simple
select query, causing it to start at normal speed and then get ever
slower. I suppose in another couple months it would take so long to run
it would never finish...
Ariel
The logging table is almost append-only*, so it is simple to make it
processed incrementally from the previous dump.
* The exception being oversight actions on old entries.