[Labs-l] large bots

[Tim Landscheidt]

Petr Bena <benapetr at gmail.com> wrote:

> in order to save system resources on bots project I decided to create one
> huge application instance instead of ton of small instances.

> If we consider the usage of operating system for disk storage, and RAM, you
> easily find out that 10 small instances eat about 1800MB RAM and  20GB of
> hdd storage more than it would be needed in case we had only one OS working
> with multiple processes.

> I know that this bring out some extra problems, so I would rather consider
> it experiment for now. But given that we install some process killers
> handling out broken bots that eat too much ram and such, it should work
> (though it would be more like toolserver used to be). Of course, some extra
> bots can still live on their own dedicated servers.

> So in case you want to help me test this new experiment and you have some
> bots running on bots-2 - bots-4 you can move them to bots-bnr1 (big non
> root = bnr)

On Toolserver, bots can use SGE to distribute the load more
evenly over the cluster
(cf. https://wiki.toolserver.org/view/Job_scheduling).  I
had asked at
http://www.mediawiki.org/wiki/Wikimedia_Labs/Toolserver_features_wanted_in_Tool_Labs
if there is a "standard" way of job scheduling in Ubuntu
shops - so is there? :-)

Tim