On Wed, 14 Jan 2004 13:52:01 -0800, Jimmy Wales wrote:
This is something we should discuss. Right now, by refusing access to people who don't send a user agent, we may be messing some people up who are coming from behind a proxy? That's what I understand him to be saying.
They're running Squid then and take the User-Agent out (header_replace User-Agent). They could as well replace it with something else of their choice instead, for example: header_replace User-Agent: Mosaic 0.1
Was there a lot of traffic from misconfigured spiders? All search engines i've come across send a header, often with url to the search engine.