On Mon, 29 Sep 2003 11:07:26 -0700, Brion Vibber brion@pobox.com gave utterance to the following:
If one of the list admins sent this through; please don't. It's *spam*.
They crawl the web for e-mail addresses and then send these things out as advertisements for their link-checking / availability service. Which, of course, means they're crawling our web site and loading it down. *Thanks* so much.
I wonder if their crawler respects robots.txt? How hard would it be to id the crawler from server logs? Does W keep server logs?