Anthony wrote:
On 10/26/07, Yousef Ourabi yourabi@zero-analog.com wrote:
What I don't understand is how this fits in with the "no-crawler" policy?
What "no-crawler" policy?
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
[[Wikipedia:Database download#Please do not use a web crawler]]
--Darkwind
Wikix downloads images in a non-intrusive manner, in fact, its no more intrusive than your average workstation browsing the site. This was by design. I intentionally make the tool slow in order to avoid any impacts on the site.
Given google's high rankings of Wikipedia pages, and the relationship with ask.com and others, it obvious that massive web crawling of the site is permitted by these search engines and in fact is encouraged.
Needless to say, wikix is nowhere near as intense as these other applications. Provided it is not being used in a malicious manner, it does not appear to impinge on this policy. Esspecially since Wikimedia states at download.wikimedia.org image tarballs will be supported at some point.
Jeff
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l