[Labs-l] Memory limits for tools

[Magog The Ogre]

Good evening, all.

I have a question: what is the maximum amount of memory that I could run
without upsetting the systems team?

I have a number of bots that run daily operations by analyzing the entire
subset of uploads on Commons on a daily basis. They do this by caching the
upload meta-data and text of the each file page.

Recently, they've been running out of memory; investigation seems to point
to there *not* being a memory leak, rather just an increase in uploads and
inefficient array storage by PHP.

It appears that I may need around 3-4GB to continue my tasks. My options
are to provide a *really* high memory limit, or to perform an extensive
code rewrite.

I'd certainly prefer the former if at all possible. :)

Thanks
Magog
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-l/attachments/20141227/5826547c/attachment.html>