Hi there,
Last year, we starting evaluating how we could refresh the way we relate (deploy, maintain, upgrade) our Openstack deployment for Cloud VPS [0].
One of the most compelling options we found was to run Openstack inside Kubernetes, using an upstream project called openstack-helm.
But... What if we stopped doing Openstack at all?
To clarify, the base idea I had is:
* deploy Kubernetes to a bunch of hosts in one of our Wikimedia datacenters ** we know how to do it! ** this would be the base, undercloud, or bedrock, whatever.
* deploy ceph next to k8s (maybe, inside even?) ** ceph would remain the preferred network storage solution
* deploy some kind of k8s multiplexing tech ** example: https://www.vcluster.com/ but there could be others ** using this create a dedicated k8s cluster for each project, for example: toolforge/toolsbeta/etc
* Inside this new VM-less toolforge, we can retain pretty much the same functionalities as today: ** a container listening on 22/tcp with kubectl & toolforge cli installed can be the login bastion ** NFS server can be run on a container, using ceph ** toolsDB can be run on a container. Can't it? Or maybe replace it with other k8s-native solution
* If we need any of the native openstack components, for example Keystone or Swift we may run them on an standalone fashion inside k8s.
* We already have some base infrastructure (and knowledge) that would support this model. We have cloudlbs, cloudgw, we know how to do ceph, etc.
* And finally, and most important: the community. The main question could be: ** Is there any software running on Cloud VPS virtual machines that cannot run on a container in kubernetes?
I wanted to start this email hoping that I would collect a list of use cases, blockers, and strong opinions about why running Openstack is important (or not). I'm pretty sure I'm overlooking some important thing.
I plan to document all this on wikitech, and/or maybe phabricator.
You may ask: and why stop doing openstack? I will answer that in a different email to keep this one as short as possible.
Looking forward to your counter-arguments. Thanks!
regards.
[0] https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/Enhancemen...