On Mon, Jan 12, 2009 at 5:35 AM, Nicolas Dumazet <nicdumz(a)gmail.com> wrote:
It seems to be a set of tools: crawler, parsers,
indexers... To allow
a search. In short, an experimental search engine. Or a wannabe
commercial engine maybe, given that a .net domain is registered:
http://www.paxle.net/
...
This tool seem to have a blacklist:
"org.paxle.filter.blacklist.impl.BlacklistFilter". If you're able to
reach the author, you can probably ask him to blacklist your tools.
Question is _how_ : I haven't been able to find an email or any
information on this.
I don't think contacting the author should be necessary. If the bot
is obeying robots.txt and other relevant directives, use those to
block it. If it's not obeying robots exclusion standards, it should
be blocked site-wide with an informative error message.