Jens Frank wrote:
Even worse than the crash recovery issue is the locking of MyISAM databases, which is not very granular. With our change rate, MyISAM is not an option.
Well... we've already established that this would have to live in a separate table, since it would have to be MyISAM. The table doesn't need to be written to for each edit, only when a category is added/changed/deleted from a page, which must be a pretty small subset of edits (wild guess: 10%?), so maybe it's not that big a problem...?
To dig up a little history, the queries I/we have tried on the existing categorylinks table take too long, so we either 1) can't run queries against really big categories or 2) need a separate table/data structure.
Aerik
On 11/12/06, Aerik Sylvan aerik@thesylvans.com wrote:
Well... we've already established that this would have to live in a separate table, since it would have to be MyISAM. The table doesn't need to be written to for each edit, only when a category is added/changed/deleted from a page, which must be a pretty small subset of edits (wild guess: 10%?), so maybe it's not that big a problem...?
Probably higher than that, to be honest, especially with all those Wikipedians running around, dealing with Special:Uncategorisedpages, and correcting stub templates and so forth.
Rob Church
Rob Church wrote:
On 11/12/06, Aerik Sylvan aerik@thesylvans.com wrote:
Well... we've already established that this would have to live in a separate table, since it would have to be MyISAM. The table doesn't need to be written to for each edit, only when a category is added/changed/deleted from a page, which must be a pretty small subset of edits (wild guess: 10%?), so maybe it's not that big a problem...?
Probably higher than that, to be honest, especially with all those Wikipedians running around, dealing with Special:Uncategorisedpages, and correcting stub templates and so forth.
Also don't forget about vandals just blanking a (categorised) page, plus the immediately following revert.
On 11/12/06, Timwi timwi@gmx.net wrote:
Also don't forget about vandals just blanking a (categorised) page, plus the immediately following revert.
Quite. In fact, I'd be willing to bet that the proportion of maintenance edits (janitorial) is increasing against normal content edits.
Rob Church
wikitech-l@lists.wikimedia.org