[Labs-l] Added 3 compute nodes to the pmtpa cluster

[Ryan Lane]

We're beginning to reach the limits of how many instances we can run on the
existing compute nodes. There's a few reasons:

1. We're getting low on disk space on a couple of nodes
2. OpenStack nova, in this version, runs periodic tasks in the same thread
as other actions and blocks for long periods of time. These tasks become
longer and longer to run as we add more instances on a host. This causes a
couple issues:
    a. It causes actions like create, reboot, get-console-output, etc. to
take a long time to be picked up and handled, in some cases completely
timing out.
    b. It causes instance creation to occasionally result in an error
state, if all hosts block long enough in periodic tasks to have their
service marked as down in the scheduler.
3. We're getting low on memory on a number of hosts

Soon I'll be moving some instances from the current hosts to the new hosts
to alleviate some of these issues.

- Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/labs-l/attachments/20130121/b2f123d2/attachment.html>