On Thu, Oct 10, 2002 at 10:28:00AM +0100, Neil Harris
wrote:
The new maintenance page is great! But the links
on it should be
disabled for non logged-in users so that web spiders don't exercise the
high-overhead functions on the maintenance page, bogging down the server.
This might be better handled with robots.txt...
I'd do both. Robots.txt only keeps out well-behaved robots, but it's the
badly-behaved robots that cause most real trouble. And if they are badly
behaved enough to hammer links, they are probably also badly behaved
enough to ignore robots.txt.
Neil