E23 wrote:
I've looked in the logs, and there are many
entries of this kind:
nnn.nnn.nnn.nnn - - [10/Nov/2003:05:06:31 +0000] "GET / HTTP/1.1" 200
18558 "http://one-specific-pornsite.net/..." "Mozilla/4.0 (compatible;
MSIE 5.01; Windows NT5.0)"
Almost every entry has different IP, and the specific sub page on the
porn site varies as well. The requested page is always "/" and the
user agent is always set exactly as above. There are never any
accompanying requests for such things as images or style sheets.
This thing clearly is a robot used for the single purpose of getting
just such links as on our stats page, to increase their google ranking.
If we blocked search engines such as Google from indexing the stats
pages, using robots.txt, that would at least have the effect of making
referrer-link spamming pointless.
-- Neil