gt;; I'm sure there are plenty of
similar tools out there.
On Sun, Feb 1, 2009 at 1:17 PM, Philip Beach <beachboy4231(a)gmail.com> wrote:
Ok thanks, but how can I be sure it's working
before all my pages drop
off google (if it were wrong). Is there some way to validate it in
the context of my site?
Thanks again
On 2/1/09, Benjamin Lees <emufarmers(a)gmail.com> wrote:
TryUser-agent: *
Disallow: /index.php
Disallow: /skins/
Disallow: /Special:Search
Disallow: /Special:Random
Some other good rules to include are
Disallow: /MediaWiki:
Disallow: /Template:
and maybe
Disallow: /Category:
On Sun, Feb 1, 2009 at 2:06 AM, Philip Beach <beachboy4231(a)gmail.com>
wrote:
> Haha true about rotting RAM, I'll look into that. I am not using
wikimedia
> style URL's, sadly :( it just didn't
happen when the site was first set
up
> and I can't move it now, for various
reasons. All of my files are in the
> web-root /. However, through an apache alias, my url is
>
mywiki.com/Pagename.
>
> How would robots.txt look for that? Would I simply drop the preceeding
> /wiki, like this?
>
> User-agent: *
> Disallow: /Special:Search
> Disallow: /Special:Random
>
> Thanks a ton!
>
> On Sun, Feb 1, 2009 at 1:47 AM, Benjamin Lees <emufarmers(a)gmail.com>
> wrote:
>
> > Assuming you have Wikimedia-style URLs:
> > User-agent: *
> > Disallow: /w/
> > Disallow: /wiki/Special:Search
> > Disallow: /wiki/Special:Random
> >
> > Your server will be able to handle a lot more if you set up as much
> caching
> > as you can <http://www.mediawiki.org/wiki/Manual:Cache>. No sense
> letting
> > all that spare RAM rot. :)
> >
> >
> > On Sat, Jan 31, 2009 at 10:02 PM, Philip Beach <
beachboy4231(a)gmail.com
>wrote:
> I already have checked the access logs. It appears that Google and
Yahoo
> are
> indeed generating a lot of traffic. Good idea Rob, I've been working
> on
> this
> for a while.
>
> Just out of curiosity, what should my robots.txt look like for
Mediawiki.
> Does anything need to be disallowed?
>
> On Sat, Jan 31, 2009 at 8:30 PM, Platonides <Platonides(a)gmail.com>
wrote:
>
> > You should check the access logs for which is causing the error.
> >
> >
> > _______________________________________________
> > MediaWiki-l mailing list
> > MediaWiki-l(a)lists.wikimedia.org
> >
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> >
> _______________________________________________
> MediaWiki-l mailing list
> MediaWiki-l(a)lists.wikimedia.org
>
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
>
_______________________________________________
MediaWiki-l mailing list
MediaWiki-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
_______________________________________________
MediaWiki-l mailing list
MediaWiki-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
_______________________________________________
MediaWiki-l mailing list
MediaWiki-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
_______________________________________________
MediaWiki-l mailing list
MediaWiki-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l