Please note that the Pywikipediabot developers have recently done some work to reduce the memory usage. More caching is used as well as the possibility to diskcache large dictionaries on disk instead of memory. I suggest that you enable this cache using "use_diskcache = True" in your user-config.py file.
Diskcache uses the cache directory in the pywikipedia folder. As the home directories are on NFS it may be faster to make a symlink from within your pywikipedia directory to /tmp:
rm -r cache mkdir /tmp/your-username-cache ln -s /tmp/your-username-cache cache
Please do not forget to update to the latest SVN version, especially if you are using interwiki.py
Bryan
Thanks Bryan ;)
A little addendum : * This feature is relatively new, and even if we have been testing it and have fixed, in particular, some platform-dependent issues - Did you read windows issues ? - some nasty bugs might still subsist. Please, please, please, do not let them be : allow us to hunt them, report them ! :) [1]
* On a performance side, I'd be very interested on any comparative ram usage reports from you, regular bot owners. Ram usage on interwiki runs has always been a problem, and it has also always been a very tricky issue to diagnose, because it depends a lot on the data (obviously, if only a couple of interwiki cycles are encountered during the run, very little memory will be used). Any comparative data on memory usage with and without diskcache would be much appreciated.
Thanks :)
[1] https://sourceforge.net/projects/pywikipediabot/ or simply #pywikipediabot on Freenode
toolserver-l@lists.wikimedia.org