Refactoring out useful components ftw!
On Mon, Feb 23, 2015 at 4:48 PM, Gabriel Wicke <gwicke(a)wikimedia.org> wrote:
in our quest to simplify the creation of production-quality services we
have recently made some modest progress with a service-runner module 
and an early service-template-node prototype . The latter deserves its
own discussion (and is at a very early stage), so I'm focusing on
service-runner in this thread.
Service-runner  is a small module that we moved out of restbase. It
generalizes some simple start-up, monitoring and supervision facilities
that we refined while building services like Parsoid, Mathoid or RESTBase:
- commandline option parsing & yaml config loading (with a standard format)
- worker pool management using the cluster module; graceful restarts
- logging via gelf to logstash, with option to configure other backends
- generic metric reporting (txstatsd, statsd or simple logging for
- general worker monitoring & debugging: V8 heap metrics & limiting,
support for heap dumps
For small (third party) installs with limited memory, we also added the
capability to cleanly run multiple services in a single process.
The uniform way to run and configure services provided by service-runner
should let us create a shared puppet module to manage most of the
per-service tasks . Another possibility is to automatically build
packages for these services , which can help to distribute these
services to third-party users.
So, please have a look & let us know what you are missing / would like to
see. There is a service-runner tag on phabricator  that we can use to
track specific tasks.
Wikitech-l mailing list