Hi all and TIA.
I usually run my bot with maxlag = 5 seconds.
When it attempts a communication with Wikipedia, if it receives Error:maxlag or HTTP 503 , it retries up to 5 times with 10 seconds between each try. If it still can't get through, it aborts. At the time I'm writing this, about 3pm Friday afternoon in Australia, my bot is routinely aborting due to 5 maxlag or 503s in a row.
I am usually logged into #wikipedia and #wikipedia-en on Freenode, and have noticed that many times I and other Australians will complain that Wikipedia is very slow and timing out for human edits, when Americans claim that they have no such problem at that time.
Currently I'm running the bot with maxlag = 10 seconds, and it's having no problems now.
Is the maxlag parameter something that will change if I'm connecting from Australia? May I be allowed some leniency for being on the other side of the world, or does it not work like that? If maxlag > 5 seconds, does that mean I should stop wasting the server's time and run my bot some other time?
Richard
I don't know why Australia might be having trouble, unless the Christchurch earthquake irritated a cable.
The way I have my bots avoid irritating the servers is to back off very quickly, and slowly speed back up. Starting with a minimum of 5 seconds between updates... If there is no reply after 30 seconds, the bot sleeps for 60 seconds before retrying. Another failure increases the sleep time to 120 seconds. A successful update decreases the sleep time by 10 seconds, until the minimum 5 seconds is reached. So a slow period can slow the bot to several minutes between attempts, and it then takes many minutes to return to normal.
On Thu, 2011-03-03 at 23:44 -0500, richardcavell@mail.com wrote:
Hi all and TIA.
I usually run my bot with maxlag = 5 seconds.
When it attempts a communication with Wikipedia, if it receives Error:maxlag or HTTP 503 , it retries up to 5 times with 10 seconds between each try. If it still can't get through, it aborts. At the time I'm writing this, about 3pm Friday afternoon in Australia, my bot is routinely aborting due to 5 maxlag or 503s in a row.
I am usually logged into #wikipedia and #wikipedia-en on Freenode, and have noticed that many times I and other Australians will complain that Wikipedia is very slow and timing out for human edits, when Americans claim that they have no such problem at that time.
Currently I'm running the bot with maxlag = 10 seconds, and it's having no problems now.
Is the maxlag parameter something that will change if I'm connecting from Australia? May I be allowed some leniency for being on the other side of the world, or does it not work like that? If maxlag > 5 seconds, does that mean I should stop wasting the server's time and run my bot some other time?
Richard
Wikibots-l mailing list Wikibots-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibots-l
"Scot Wilcoxon" scot@wilcoxon.org wrote:
The way I have my bots avoid irritating the servers is to back off very quickly, and slowly speed back up. Starting with a minimum of 5 seconds between updates... If there is no reply after 30 seconds, the bot sleeps for 60 seconds before retrying. Another failure increases the sleep time to 120 seconds. A successful update decreases the sleep time by 10 seconds, until the minimum 5 seconds is reached. So a slow period can slow the bot to several minutes between attempts, and it then takes many minutes to return to normal.
On Thu, 2011-03-03 at 23:44 -0500, richardcavell@mail.com wrote:
Hi all and TIA.
I usually run my bot with maxlag = 5 seconds.
When it attempts a communication with Wikipedia, if it receives Error:maxlag or HTTP 503 , it retries up to 5 times with 10 seconds between each try. If it still can't get through, it aborts. At the time I'm writing this, about 3pm Friday afternoon in Australia, my bot is routinely aborting due to 5 maxlag or 503s in a row.
The algorithm I've used in working on pywikipediabot is, upon receiving a maxlag error, to have the bot sleep for min(5, lag / 2) seconds, where 'lag' is the lag reported in the error message. If there is a long replication lag, then it is going to take longer for the database slaves to catch up than with a short lag, so there is no point in rechecking every 5 seconds. Aborting after 5 maxlag errors is purely a design choice; if you don't mind having your bot wait minutes or hours for the lag to subside, you can do that too.
Russ
richardcavell@mail.com wrote:
Hi all and TIA.
I usually run my bot with maxlag = 5 seconds.
When it attempts a communication with Wikipedia, if it receives Error:maxlag or HTTP 503 , it retries up to 5 times with 10 seconds between each try. If it still can't get through, it aborts. At the time I'm writing this, about 3pm Friday afternoon in Australia, my bot is routinely aborting due to 5 maxlag or 503s in a row.
I am usually logged into #wikipedia and #wikipedia-en on Freenode, and have noticed that many times I and other Australians will complain that Wikipedia is very slow and timing out for human edits, when Americans claim that they have no such problem at that time.
Currently I'm running the bot with maxlag = 10 seconds, and it's having no problems now.
Is the maxlag parameter something that will change if I'm connecting from Australia? May I be allowed some leniency for being on the other side of the world, or does it not work like that? If maxlag > 5 seconds, does that mean I should stop wasting the server's time and run my bot some other time?
Richard
The lag only affects the server themselves. It just means that at that time the lag was between 5 and 10 seconds for several minutes.
You probably wan to take a look at http://en.wikipedia.org/w/api.php?action=query&meta=siteinfo&siprop=... to see what's going on.
wikibots-l@lists.wikimedia.org