Hi folks, i've been running a bot on the toolserver and what normally occurs is that the process is killed for the excessive usage of memory.
Is there a method to limit the memory that a process have? So i am not rerunning it once every day?
( The process seems to work fine without using too much memory for some hours)
thanks for any help
Davide / Helios
Davide writes:
Hi folks, i've been running a bot on the toolserver and what normally occurs is that the process is killed for the excessive usage of memory.
Is there a method to limit the memory that a process have? So i am not rerunning it once every day?
( The process seems to work fine without using too much memory for some hours)
thanks for any help
Davide / Helios
Yes, release (or reuse) memory --VasilievVV
On Fri, 2007-12-28 at 15:52 +0100, Davide wrote:
Well the purpose of that mail was to ask if there is a command i should use, e.g. ulimit, to limit the memory of my processes. Davide
If your process runs out of memory, even due to a per-process basis, I presume it will simply segfault and die, and you will have the same problem you are having now. The only way I know to avoid requesting more memory than is available (which is what causes the process to die) is to rework your code to be more memory-efficient.
-B.
On the mentioning of that, interwiki.py consumes too much memory. Anyone could take a look at it and make it more memory friendly?..
--user:alnokta
Davide:
D> Is there a method to limit the memory that a process have? So i am not D> rerunning it once every day?
you can't give a process less memory than it needs, it'll stop working. in this case, i suggest asking the pywikipedia developers to make their programs use a bit less memory.
- river.
An interesting trick that I use is having two scripts, one with all the bot code and one script that runs the other script in a loop. This is very good if your bot needs to be run on the hour and it stops the bot code from taking up too much memory.
Chris
On Dec 28, 2007 7:51 PM, Christopher Grant chrisgrantmail@gmail.com wrote:
An interesting trick that I use is having two scripts, one with all the bot code and one script that runs the other script in a loop. This is very good if your bot needs to be run on the hour and it stops the bot code from taking up too much memory.
Chris
Toolserver-l mailing list Toolserver-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/toolserver-l
Wouldn't a cronjob give the same effect?
Wouldn't a cronjob give the same effect?
If you have a script that runs, say, on the hour, one benefit of doing a loop instead of a cron job is that if your main script unexpectedly fails to exit, or takes longer than an hour to execute, you don't end up creating a large number of copies of the same script on the machine.
-B.
On 12/28/07, Christopher Beland beland@alum.mit.edu wrote:
Wouldn't a cronjob give the same effect?
If you have a script that runs, say, on the hour, one benefit of doing a loop instead of a cron job is that if your main script unexpectedly fails to exit, or takes longer than an hour to execute, you don't end up creating a large number of copies of the same script on the machine.
You can also add a simple check at the beginning looking for duplicate copies running on the same machine, and take appropriate action (either dying, or killing it, whichever you prefer). There's not much need to reinvent cron, either way.
toolserver-l@lists.wikimedia.org