In CVS:
After some indirect prompting from Erik, I wrote some link table update code which hopefully will speed up edits to pages with lots of links. Basic features:
* fairly well enabled/disabled with $wgUseBetterLinksUpdate * Article::showArticle() loads the link table with LinkCache::preFill(), then calls LinkCache::clear(), which clears the cache but leaves a copy of it hidden away. It then refills the cache with the new link table. * LinksUpdate::doUpdate() calls LinkCache::incrementalSetup(), which works out what needs to be added and deleted, and applies a simple formula to decide whether the old way or the new way should be quicker. * The new way deletes links row by row, and then adds the new ones using a single insert. Blanking large articles would be slow, hence that would be done the old way.
This has not been tested for speed yet. I think constructing a realistic test would be difficult -- it's probably easier just to put it on the live server and see what happens.
-- Tim Starling.
Tim Starling wrote:
After some indirect prompting from Erik, I wrote some link table update code which hopefully will speed up edits to pages with lots of links.
Yay!
- fairly well enabled/disabled with $wgUseBetterLinksUpdate
- Article::showArticle() loads the link table with LinkCache::preFill(),
then calls LinkCache::clear(), which clears the cache but leaves a copy of it hidden away. It then refills the cache with the new link table.
(Haven't looked at it in much detail yet.) Hmm, does this mean we take the overhead of a double cache on every page view? We should only need it when saving pages. I suppose in most cases it's the least of our troubles though. :)
- LinksUpdate::doUpdate() calls LinkCache::incrementalSetup(), which works
out what needs to be added and deleted, and applies a simple formula to decide whether the old way or the new way should be quicker.
- The new way deletes links row by row, and then adds the new ones using a
single insert. Blanking large articles would be slow, hence that would be done the old way.
It may be a touch more efficient to do the deletes in one query, such as: $sql = "DELETE FROM links WHERE l_from='{$this->mTitleEnc}' AND l_to IN (" . implode( ",", $del) . ")";
-- brion vibber (brion @ pobox.com)
"Brion Vibber" brion@pobox.com wrote in message news:3F081435.1070100@pobox.com...
Tim Starling wrote:
...
- Article::showArticle() loads the link table with LinkCache::preFill(),
then calls LinkCache::clear(), which clears the cache but leaves a copy
of
it hidden away. It then refills the cache with the new link table.
(Haven't looked at it in much detail yet.) Hmm, does this mean we take the overhead of a double cache on every page view? We should only need it when saving pages. I suppose in most cases it's the least of our troubles though. :)
No, showArticle() is only called from insertNewArticle() and updateArticle(). In fact, it doesn't show anything at all, it just updates some tables and sends a redirect.
- LinksUpdate::doUpdate() calls LinkCache::incrementalSetup(), which
works
out what needs to be added and deleted, and applies a simple formula to decide whether the old way or the new way should be quicker.
- The new way deletes links row by row, and then adds the new ones using
a
single insert. Blanking large articles would be slow, hence that would be done the old way.
It may be a touch more efficient to do the deletes in one query, such as: $sql = "DELETE FROM links WHERE l_from='{$this->mTitleEnc}' AND l_to IN (" . implode( ",", $del) . ")";
I'll take your word for it. MySQL is still pretty much a mystery to me.
-- Tim Starling.
06.07.2003 14:21:09, Brion Vibber brion@pobox.com wrote:
It may be a touch more efficient to do the deletes in one query, such as: $sql = "DELETE FROM links WHERE l_from='{$this->mTitleEnc}' AND l_to IN (". implode( ",", $del) . ")";
Probably not just a touch. :-)
Yeah, I was going to say the same. Do it this way, and then this way should always be faster than the old way, so trash it.
I would have loved to do it, but I guess I'm too late :-) Timwi
Tim-
thanks for coding this. One bug I noticed: When creating and removing a link to a non-existent page, the brokenlinks table is not updated accordingly, i.e. the removed link is still there. It works fine for normal links.
Regards,
Erik
"Erik Moeller" erik_moeller@gmx.de wrote in message news:8pOynqihpVB@erik_moeller...
Tim-
thanks for coding this. One bug I noticed: When creating and removing a link to a non-existent page, the brokenlinks table is not updated accordingly, i.e. the removed link is still there. It works fine for normal links.
Done. It was an unquoted string in the SQL query which went unreported and unlogged, since database errors get lost when a redirect is sent.
"Timwi" timwi@gmx.net wrote in message news:TNJFZUQN1VNKJHMK97IGWTSMNHONB63Y.3f08239c@xptoe...
06.07.2003 14:21:09, Brion Vibber
brion@pobox.com wrote:
It may be a touch more efficient to do the deletes in one query, such as: $sql = "DELETE FROM links WHERE l_from='{$this->mTitleEnc}' AND l_to IN (". implode( ",", $del) . ")";
Probably not just a touch. :-)
Yeah, I was going to say the same. Do it this way, and then this way should always be faster than the old way, so trash it.
I would have loved to do it, but I guess I'm too late :-)
You just want me to do all the work, don't you? :) Done.
-- Tim Starling.
wikitech-l@lists.wikimedia.org