Taku made a suggestion I've been mulling over:
Can we find out how much performance is affected by checking for broken links?
I personally don't care most of the time whether a link is "real" or "broken". It's only when I'm actively editing an article that I need to know this.
And anyway, isn't there a way to CACHE some of this real/broken info in my browser?
Take what I did yesterday, on one of my rare weekend editing sessions. I was trying to straighten out links and naming for a few of the recent mikado (i.e., Japanese emperors). I kept having to refresh the page to check whether certain links were present, simply because I couldn't remember. But I bet my browser knew.
Anyway, before I go off half-cocked, let me ask this:
What percent of processor time (or disk access time or whatever) goes into maintaining the redness or blueness of all those links?
A) If it's a tiny percentage, then forget I ever posted this (return from interrupt!) B) If it's more than half, I'd say we better look into this.
Give the user an option to turn off broken-link-checking, when they so choose? Only if it would remove an appreciable amount of load from the server.
----
Sorry, but I didn't have time to polish this draft (time is pressing), but I hope you get the idea: look for where the bulk of the time goes, and try to reduce that time expenditure.
Uncle Ed Lazy Developer, Wikipedia