Okay, I have ruled out bots. After disallowing all user-agents for a few days the load did not decrease, my site continued having errors. I have also, with the help of my host, ruled out DOS attacks. I have also optimized the database. I don't know what else to try, my users are still getting this error every couple of minutes.
Service Temporarily Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
It seems when analyzing the load factor compared to the time the errors appear that the load steadily increases to a factor of about 25, then the error appears, the load decreases, and it goes back online. My host said this was the way that the server behaves when under a heavy load of legit users and that I should focus on optimizing the database. However, this is a brand new quad-core server. My old server (dual core, half the ram) was running the same exact database with no problems.
Does anyone have any ideas?
On Mon, Feb 2, 2009 at 3:34 AM, Benjamin Lees emufarmers@gmail.com wrote:
Google offers such a tool with their Webmaster Tools < https://www.google.com/webmasters/tools%3E; I'm sure there are plenty of similar tools out there.
On Sun, Feb 1, 2009 at 1:17 PM, Philip Beach beachboy4231@gmail.com wrote:
Ok thanks, but how can I be sure it's working before all my pages drop off google (if it were wrong). Is there some way to validate it in the context of my site?
Thanks again
On 2/1/09, Benjamin Lees emufarmers@gmail.com wrote:
TryUser-agent: * Disallow: /index.php Disallow: /skins/ Disallow: /Special:Search Disallow: /Special:Random
Some other good rules to include are Disallow: /MediaWiki: Disallow: /Template:
and maybe Disallow: /Category:
On Sun, Feb 1, 2009 at 2:06 AM, Philip Beach beachboy4231@gmail.com
wrote:
Haha true about rotting RAM, I'll look into that. I am not using
wikimedia
style URL's, sadly :( it just didn't happen when the site was first
set
up
and I can't move it now, for various reasons. All of my files are in
the
web-root /. However, through an apache alias, my url is mywiki.com/Pagename.
How would robots.txt look for that? Would I simply drop the preceeding /wiki, like this?
User-agent: * Disallow: /Special:Search Disallow: /Special:Random
Thanks a ton!
On Sun, Feb 1, 2009 at 1:47 AM, Benjamin Lees emufarmers@gmail.com wrote:
Assuming you have Wikimedia-style URLs: User-agent: * Disallow: /w/ Disallow: /wiki/Special:Search Disallow: /wiki/Special:Random
Your server will be able to handle a lot more if you set up as much
caching
as you can http://www.mediawiki.org/wiki/Manual:Cache. No sense
letting
all that spare RAM rot. :)
On Sat, Jan 31, 2009 at 10:02 PM, Philip Beach <
beachboy4231@gmail.com
wrote:
I already have checked the access logs. It appears that Google and
Yahoo
are indeed generating a lot of traffic. Good idea Rob, I've been
working
on this for a while.
Just out of curiosity, what should my robots.txt look like for
Mediawiki.
Does anything need to be disallowed?
On Sat, Jan 31, 2009 at 8:30 PM, Platonides <Platonides@gmail.com
wrote:
> You should check the access logs for which is causing the error. > > > _______________________________________________ > MediaWiki-l mailing list > MediaWiki-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l > _______________________________________________ MediaWiki-l mailing list MediaWiki-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
MediaWiki-l mailing list MediaWiki-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
MediaWiki-l mailing list MediaWiki-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
MediaWiki-l mailing list MediaWiki-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
MediaWiki-l mailing list MediaWiki-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
MediaWiki-l mailing list MediaWiki-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-l