Bugs item #2114223, was opened at 2008-09-16 15:46
Message generated for change (Comment added) made by nicdumz
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=211422…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: interwiki
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Andr Malafaya Baptista (malafaya)
Assigned to: Nobody/Anonymous (nobody)
Summary: Socket timeout breaks out
Initial Comment:
VERSION.PY
==========
Pywikipedia [svn+ssh] wikimedia/svnroot/pywikipedia/trunk/pywikipedia (r5898, Se
p 16 2008, 11:50:17)
Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)]
DESCRIPTION
===========
It's been happening in the past days that a socket timeout interrupts the bot. I
believe the stack trace below is self-explanatory.
I used the command line:
interwiki.py -family:wiktionary -autonomous -start:Category:! -lang:io
OUTPUT
======
NOTE: The first unfinished subject is [[io:Kategorio:Albaniana vorti]]
NOTE: Number of pages queued is 59, trying to add 60 more.
Sleeping for 4.1 seconds, 2008-09-16 14:31:06
Dump io (wiktionary) saved
Traceback (most recent call last):
File "D:\Work\pywikipediabot-HEAD\pywikipedia\interwiki.py", line 1735, in
<module>
bot.run()
File "D:\Work\pywikipediabot-HEAD\pywikipedia\interwiki.py", line 1486, in
run
self.queryStep()
File "D:\Work\pywikipediabot-HEAD\pywikipedia\interwiki.py", line 1460, in
queryStep
self.oneQuery()
File "D:\Work\pywikipediabot-HEAD\pywikipedia\interwiki.py", line 1428, in
oneQuery
site = self.selectQuerySite()
File "D:\Work\pywikipediabot-HEAD\pywikipedia\interwiki.py", line 1402, in
selectQuerySite
self.generateMore(globalvar.maxquerysize - mycount)
File "D:\Work\pywikipediabot-HEAD\pywikipedia\interwiki.py", line 1336, in
generateMore
page = self.pageGenerator.next()
File "D:\Work\pywikipediabot-HEAD\pywikipedia\pagegenerators.py", line 688,
in
DuplicateFilterPageGenerator
for page in generator:
File "D:\Work\pywikipediabot-HEAD\pywikipedia\pagegenerators.py", line 239,
in
AllpagesPageGenerator
for page in site.allpages(start = start, namespace = namespace, includeredir
ects = includeredirects):
File "D:\Work\pywikipediabot-HEAD\pywikipedia\wikipedia.py", line 5166, in
allpages
text = self.getUrl(api_url)
File "D:\Work\pywikipediabot-HEAD\pywikipedia\wikipedia.py", line 4485, in
getUrl
text = f.read()
File "D:\Program Files\Python\lib\socket.py", line 291, in read
data = self._sock.recv(recv_size)
socket.timeout: timed out
----------------------------------------------------------------------
Comment By: NicDumZ Nicolas Dumazet (nicdumz)
Date: 2008-09-20 03:41
Message:
the pagegenerator, even with the new api implementation, seems to be
working, I'm currently listing the pages of eo.wikt without any timeout.
Your connection might just be slower than usual ? Or does it timeout when
the WM websites are under heavy load ?
You can tweak the socket timeout in user-config.py, setting socket_timeout
to the number of seconds to wait (default is 120 seconds, quite long...)
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=211422…