If this is the problem, we are in luck because there have been a lot of good improvement suggestions. But they all add complexity to the code (or database setup) and "premature optimization is the root of all kinds of evil," so if link checking isn't a bottleneck it would
be
counterproductive to spend a lot of time to try to optimize it.
I think it's a bit premature for that yet. I think the differently- rendered missing links feature is pretty critical, not just a frill. In the short term the hardware will bail us out until we can find a solution.
What I was suggesting is just that we '''test''' the performance with differently rendered links turned off, so that we know if this is a place where optimization is required. I expect it is, but I think it makes sense to check.
Even a simple hash in shared means that we are maintaining data in two places all the time and thus and makes things somewhat more complex to maintain.
I think this is a good idea -- if link checking is creating the current bottleneck. And that's why I voiced support turning off the links to see the performance impact before anybody spends time coding an optimized link checking routine.
--Mark Christensen