[Mediawiki-l] robots.txt

Kasimir Gabert kasimir.g at gmail.com
Mon Oct 2 00:32:31 UTC 2006


Hello,

Excluding index.php using robots.txt should work if an article link on
your page is http://mysite.tld/My_Page.  The robots would then not
crawl http://mysite.tld/index.php?title=My_Page&action=edit, etc.

I hope that this helps,
Kasimir

On 10/1/06, Sy Ali <sy1234 at gmail.com> wrote:
> On 9/25/06, Roger Chrisman <roger at rogerchrisman.com> wrote:
> > But in the interest of short URLs, I serve my MediaWiki directly from
> > site / without any /wiki/ or /w/ directories. So above meathod would
> > not work on my installation.
> >
> > Any ideas how I can exclude robots from crawling all my wiki's edit,
> > history, talk, etc, pages *without* excluding its article pages?
>
> I do the same thing, and I never did figure out the rules to disallow
> the other sub-pages.
>
> As I understand, there are "nofol" tags within the web pages itself,
> but I'm not certain that's being honoured.
> _______________________________________________
> MediaWiki-l mailing list
> MediaWiki-l at Wikimedia.org
> http://mail.wikipedia.org/mailman/listinfo/mediawiki-l
>


-- 
Kasimir Gabert



More information about the MediaWiki-l mailing list