Hi there,
Last year, we starting evaluating how we could refresh the way we relate
(deploy, maintain, upgrade) our Openstack deployment for Cloud VPS [0].
One of the most compelling options we found was to run Openstack inside
Kubernetes, using an upstream project called openstack-helm.
But... What if we stopped doing Openstack at all?
To clarify, the base idea I had is:
* deploy Kubernetes to a bunch of hosts in one of our Wikimedia datacenters
** we know how to do it!
** this would be the base, undercloud, or bedrock, whatever.
* deploy ceph next to k8s (maybe, inside even?)
** ceph would remain the preferred network storage solution
* deploy some kind of k8s multiplexing tech
** example: https://www.vcluster.com/ but there could be others
** using this create a dedicated k8s cluster for each project, for example:
toolforge/toolsbeta/etc
* Inside this new VM-less toolforge, we can retain pretty much the same
functionalities as today:
** a container listening on 22/tcp with kubectl & toolforge cli installed can be
the login bastion
** NFS server can be run on a container, using ceph
** toolsDB can be run on a container. Can't it? Or maybe replace it with other
k8s-native solution
* If we need any of the native openstack components, for example Keystone or
Swift we may run them on an standalone fashion inside k8s.
* We already have some base infrastructure (and knowledge) that would support
this model. We have cloudlbs, cloudgw, we know how to do ceph, etc.
* And finally, and most important: the community. The main question could be:
** Is there any software running on Cloud VPS virtual machines that cannot run
on a container in kubernetes?
I wanted to start this email hoping that I would collect a list of use cases,
blockers, and strong opinions about why running Openstack is important (or not).
I'm pretty sure I'm overlooking some important thing.
I plan to document all this on wikitech, and/or maybe phabricator.
You may ask: and why stop doing openstack? I will answer that in a different
email to keep this one as short as possible.
Looking forward to your counter-arguments.
Thanks!
regards.
[0]
https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/Enhanceme…