On 15 January 2012 00:32, Nickanc Wikipedia nickanc.wiki@gmail.com wrote:
In my opinion, the problem is not how many bots are working, but that interwiki.py seems to use to overuse memory space. For example, my interwiki.py was running, it was spending 982 megabyte and it was killed. You may find a verbose log of its work at http://toolserver.org/~nickanc/interwiki.log .
No, that is not the problem. Multichill was referring to CPU usage, not
memory usage. And although interwiki.py in general is using a large amount of memory, your specific case has a different origin (being the use of the ReferringPageGenerator, which results in a memory *leak*).
You may be able to partially mitigate your problem by using interwiki_contents_on_disk = True, but this will not solve the actual memory leak - it will only release memory used by the page contents.
If you'd like to discuss details on your problem, please mail to the pywikipedia mailing list pywikipedia-l@lists.wikimedia.org or visit on IRC.
Merlijn