Okay, I have ruled out bots. After disallowing all user-agents for a few
days the load did not decrease, my site continued having errors. I have
also, with the help of my host, ruled out DOS attacks. I have also optimized
the database. I don't know what else to try, my users are still getting this
error every couple of minutes.
Service Temporarily Unavailable
The server is temporarily unable to service your request due to maintenance
downtime or capacity problems. Please try again later.
Additionally, a 404 Not Found error was encountered while trying to use an
ErrorDocument to handle the request.
It seems when analyzing the load factor compared to the time the errors
appear that the load steadily increases to a factor of about 25, then the
error appears, the load decreases, and it goes back online. My host said
this was the way that the server behaves when under a heavy load of legit
users and that I should focus on optimizing the database. However, this is a
brand new quad-core server. My old server (dual core, half the ram) was
running the same exact database with no problems.
Does anyone have any ideas?
On Mon, Feb 2, 2009 at 3:34 AM, Benjamin Lees <emufarmers(a)gmail.com> wrote:
Google offers such a tool with their Webmaster Tools
<
https://www.google.com/webmasters/tools>gt;; I'm sure there are plenty of
similar tools out there.
On Sun, Feb 1, 2009 at 1:17 PM, Philip Beach <beachboy4231(a)gmail.com>
wrote:
Ok thanks, but how can I be sure it's working
before all my pages drop
off google (if it were wrong). Is there some way to validate it in
the context of my site?
Thanks again
On 2/1/09, Benjamin Lees <emufarmers(a)gmail.com> wrote:
TryUser-agent: *
Disallow: /index.php
Disallow: /skins/
Disallow: /Special:Search
Disallow: /Special:Random
Some other good rules to include are
Disallow: /MediaWiki:
Disallow: /Template:
and maybe
Disallow: /Category:
On Sun, Feb 1, 2009 at 2:06 AM, Philip Beach <beachboy4231(a)gmail.com>
wrote:
> Haha true about rotting RAM, I'll look into that. I am not using
wikimedia
>> style URL's, sadly :( it just didn't happen when the site was first
set
up
>> and I can't move it now, for various reasons. All of my files are in
the
>
web-root /. However, through an apache alias, my url is
>
mywiki.com/Pagename.
>
> How would robots.txt look for that? Would I simply drop the preceeding
> /wiki, like this?
>
> User-agent: *
> Disallow: /Special:Search
> Disallow: /Special:Random
>
> Thanks a ton!
>
> On Sun, Feb 1, 2009 at 1:47 AM, Benjamin Lees <emufarmers(a)gmail.com>
> wrote:
>
> > Assuming you have Wikimedia-style URLs:
> > User-agent: *
> > Disallow: /w/
> > Disallow: /wiki/Special:Search
> > Disallow: /wiki/Special:Random
> >
> > Your server will be able to handle a lot more if you set up as much
> caching
> > as you can <http://www.mediawiki.org/wiki/Manual:Cache>. No sense
> letting
> > all that spare RAM rot. :)
> >
> >
> > On Sat, Jan 31, 2009 at 10:02 PM, Philip Beach <
beachboy4231(a)gmail.com
>> > >wrote:
>> >
>> > > I already have checked the access logs. It appears that Google and
>> Yahoo
>> > > are
>> > > indeed generating a lot of traffic. Good idea Rob, I've been
working
> >
> on
> > > this
> > > for a while.
> > >
> > > Just out of curiosity, what should my robots.txt look like for
> Mediawiki.
> > > Does anything need to be disallowed?
> > >
> > > On Sat, Jan 31, 2009 at 8:30 PM, Platonides <Platonides(a)gmail.com
>
wrote:
> >
> > > You should check the access logs for which is causing the error.
> > >
> > >
> > > _______________________________________________
> > > MediaWiki-l mailing list
> > > MediaWiki-l(a)lists.wikimedia.org
> > >
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> > >
> > _______________________________________________
> > MediaWiki-l mailing list
> > MediaWiki-l(a)lists.wikimedia.org
> >
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
> >
> _______________________________________________
> MediaWiki-l mailing list
> MediaWiki-l(a)lists.wikimedia.org
>
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
>
_______________________________________________
MediaWiki-l mailing list
MediaWiki-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
_______________________________________________
MediaWiki-l mailing list
MediaWiki-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
_______________________________________________
MediaWiki-l mailing list
MediaWiki-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
_______________________________________________
MediaWiki-l mailing list
MediaWiki-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-l