Kasimir Gabert wrote:
Hello,
Excluding index.php using robots.txt should work if an article link on your page is http://mysite.tld/My_Page. The robots would then not crawl http://mysite.tld/index.php?title=My_Page&action=edit, etc.
Kasimir, I believe you have written above a beautiful solution for my need. My article links on my site (http://wikigogy.org) are indeed done without reference to index.php but the 'edit', 'history' and other action pages that I wish to exclude are done with that reference. I had not realized this simple elegant solution. I will try it. It should look like this in my-wiki/robots.txt, right?:
User-agent: * Disallow: index.php*
Is the asterisk on index.php* correct and needed?
Thank you, Roger
On 10/1/06, Sy Ali sy1234@gmail.com wrote:
On 9/25/06, Roger Chrisman roger@rogerchrisman.com wrote:
But in the interest of short URLs, I serve my MediaWiki directly from site / without any /wiki/ or /w/ directories. So above meathod would not work on my installation.
Any ideas how I can exclude robots from crawling all my wiki's edit, history, talk, etc, pages *without* excluding its article pages?
I do the same thing, and I never did figure out the rules to disallow the other sub-pages.
As I understand, there are "nofol" tags within the web pages itself, but I'm not certain that's being honoured. _______________________________________________ MediaWiki-l mailing list MediaWiki-l@Wikimedia.org http://mail.wikipedia.org/mailman/listinfo/mediawiki-l