* Platonides Platonides@gmail.com [Wed, 16 Feb 2011 00:40:38 +0100]:
MZMcBride wrote:
To this end, I filed a JIRA ticket with the Toolserver ops about
getting the
Wikimedia Bugzilla database replicated with public views: https://jira.toolserver.org/browse/TS-901
The bugzilla db is on a different machine, it may not be worth it.
Leo wrote:
One thing that bugged me a while: Why does the bz robots.txt deny any bots? I think having it indexed
by
google & co would'nt exactly make searching harder.
Leo
A crawler can traverse fast the whole install, which means rendering a lot of dynamic pages, which bugzilla may not be able to handle.
There is Crawl-delay directive in robots.txt. And one may deny the robots which do not obey it. Dmitriy