On Thu, May 2, 2019 at 12:25 PM Andrew Bogott abogott@wikimedia.org wrote:
The following is largely notes for my future self, but Brooke might be interested in reading up about Rook.
Ceph/Rook:
Everyone is using ceph! Everyone also talks a lot about how hard it is to deploy.
Yeah, sounds about right...
There's a fair amount of buzz around 'Rook'
I met the maintainers at KubeCon and chatted with them a bit. The biggest drawback I've seen to using it as a ceph deployment was that it isn't typically accessible outside of Kubernetes without using host networking, which didn't seem an option the maintainers were terribly excited about at the time. After following some drama online around that, and figuring the additional burden of configuring a modern-enough Kubernetes on top of Ceph might slow it all down, I'd written it off at that point. They did hit 1.0 four days ago, which is pretty awesome, and it might have become less finicky despite the need for host networking (most issues around that have been closed). Also on the other hand, since I've been in the weeds digging around in requirements for Ceph, I now know that Ceph has issues around Debian kernels that make containers sound absolutely spectacular for deployment at this point (though we'll still have kernel issues on the client side...and if we cannot use the official docker images). Apparently Rook even deploys Nautilus :) We'd be ultra-modern! I'm game for trying it during the PoC at very least. At this point, the production Kubernetes versions are high enough to use it. :)
On the packaging and deploying of OpenStack, kubernetes does sound very "aligned" with other things. Maybe we check if it is horrible for storage first :) We already know what a headache k8s can be, depending on how certs and other things are managed.
Interesting stuff, thanks! When I finally get out of Cleveland, I'll have to write something up for PyCon.