FYI
---------- Forwarded message ---------- From: Yuvi Panda yuvipanda@gmail.com Date: Wed, Sep 16, 2015 at 6:25 PM Subject: [Tools] Kubernetes picked to provide alternative to GridEngine To: labs-announce@lists.wikimedia.org
We have picked kubernetes.io to provide an alternative to GridEngine on Tool Labs! See https://phabricator.wikimedia.org/T106475 for more details about the evaluation itself, https://docs.google.com/spreadsheets/d/1YkVsd8Y5wBn9fvwVQmp9Sf8K9DZCqmyJ-ew-... for the spreadsheet with the evaluation matrix and https://lists.wikimedia.org/pipermail/labs-l/2015-August/003955.html for the previous email listing pros and cons of the others solutions that we have considered.
== Rough Timeline ==
* October: Start beta testing by opening up Kubernetes to whitelisted tools. Allows people to run arbitrary Docker images in Kubernetes, both for continuous jobs and web services. If you are interested in this,please add a comment to https://phabricator.wikimedia.org/T112824 * December: Work on migrational tooling to assist in switching from GridEngine to Kubernetes should be in a good place. This will begin with a '--provider=kubernetes' parameter to the webservice command that will allow people to easily switch to Kubernetes for webservice. We will have something similar implemented for jsub / jstart. * January: Kubernetes cluster is opened up for all tools users. Fairly complete switching between GridEngine and Kubernetes for at least continuous jobs and webservices.
== What does this give developers? ==
# A software stack that is made up of widely-used and actively-developed software components. We are looking to dramatically reduce the surface of code that we write and maintain in-house. # (When out of beta) More stability and reliability. # It allows us to get rid of a lot of our customizations (the entire webservice setup, for example) which has proven to be a lot of work and flaky at times. # We can migrate tools that don't require NFS off of it, and since it has historically been one of our flakiest servies this allows tools to opt-out of it and get a lot more reliability. # Using Docker allows more user freedom in a structured way that is easier to support. # Has an active upstream / is used elsewhere as well (Google Container Engine, Microsoft Azure, RedHat OpenShift, AWS, etc.), so much more help / community available when users run into issues than we have now # Probably more :) It opens up a lot of possibilities and I'm sure developers will use it createively :)
== Do I need to change anything? ==
No, especially if you do not want to. We think Docker and Kubernetes are exciting technologies, so we would like volunteers who are interested in exploring these platforms to have the option of getting their hands dirty. At the same time, it is important for us to avoid complicating life for people for whom the current setup works well. If we are successful, the migration to Docker and Kubernetes will be seamless, and users of Tool Labs will not have to know what either Docker or Kubernetes are, let alone how to operate them.
We will begin by offering arbitrary Docker image execution only. Eventually we will make it super-easy to switch between the current setup and Kubernetes, and allow people to be able to use this without having to know what Docker is or what Kubernetes is. That will slowly be built over the next few months.
Absolutely nothing changes for developers or users right now.
== Will GridEngine be going away? ==
Not anytime soon!
Thanks!
-- Yuvi Panda T http://yuvi.in/blog
Congrats on moving forward with a big decision! I'm very optimistic about containers, so it's exciting to see movement in this area.
Is there a larger arc of using this for our own services (Mediawiki, RESTBase, etc.), potentially in production?
On Wed, Sep 16, 2015 at 9:28 PM, Yuvi Panda yuvipanda@gmail.com wrote:
FYI
---------- Forwarded message ---------- From: Yuvi Panda yuvipanda@gmail.com Date: Wed, Sep 16, 2015 at 6:25 PM Subject: [Tools] Kubernetes picked to provide alternative to GridEngine To: labs-announce@lists.wikimedia.org
We have picked kubernetes.io to provide an alternative to GridEngine on Tool Labs! See https://phabricator.wikimedia.org/T106475 for more details about the evaluation itself,
https://docs.google.com/spreadsheets/d/1YkVsd8Y5wBn9fvwVQmp9Sf8K9DZCqmyJ-ew-... for the spreadsheet with the evaluation matrix and https://lists.wikimedia.org/pipermail/labs-l/2015-August/003955.html for the previous email listing pros and cons of the others solutions that we have considered.
== Rough Timeline ==
- October: Start beta testing by opening up Kubernetes to whitelisted
tools. Allows people to run arbitrary Docker images in Kubernetes, both for continuous jobs and web services. If you are interested in this,please add a comment to https://phabricator.wikimedia.org/T112824
- December: Work on migrational tooling to assist in switching from
GridEngine to Kubernetes should be in a good place. This will begin with a '--provider=kubernetes' parameter to the webservice command that will allow people to easily switch to Kubernetes for webservice. We will have something similar implemented for jsub / jstart.
- January: Kubernetes cluster is opened up for all tools users. Fairly
complete switching between GridEngine and Kubernetes for at least continuous jobs and webservices.
== What does this give developers? ==
# A software stack that is made up of widely-used and actively-developed software components. We are looking to dramatically reduce the surface of code that we write and maintain in-house. # (When out of beta) More stability and reliability. # It allows us to get rid of a lot of our customizations (the entire webservice setup, for example) which has proven to be a lot of work and flaky at times. # We can migrate tools that don't require NFS off of it, and since it has historically been one of our flakiest servies this allows tools to opt-out of it and get a lot more reliability. # Using Docker allows more user freedom in a structured way that is easier to support. # Has an active upstream / is used elsewhere as well (Google Container Engine, Microsoft Azure, RedHat OpenShift, AWS, etc.), so much more help / community available when users run into issues than we have now # Probably more :) It opens up a lot of possibilities and I'm sure developers will use it createively :)
== Do I need to change anything? ==
No, especially if you do not want to. We think Docker and Kubernetes are exciting technologies, so we would like volunteers who are interested in exploring these platforms to have the option of getting their hands dirty. At the same time, it is important for us to avoid complicating life for people for whom the current setup works well. If we are successful, the migration to Docker and Kubernetes will be seamless, and users of Tool Labs will not have to know what either Docker or Kubernetes are, let alone how to operate them.
We will begin by offering arbitrary Docker image execution only. Eventually we will make it super-easy to switch between the current setup and Kubernetes, and allow people to be able to use this without having to know what Docker is or what Kubernetes is. That will slowly be built over the next few months.
Absolutely nothing changes for developers or users right now.
== Will GridEngine be going away? ==
Not anytime soon!
Thanks!
-- Yuvi Panda T http://yuvi.in/blog
-- Yuvi Panda T http://yuvi.in/blog
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Thu, Sep 17, 2015 at 4:00 PM, Brian Gerstle bgerstle@wikimedia.org wrote:
Congrats on moving forward with a big decision! I'm very optimistic about containers, so it's exciting to see movement in this area.
Is there a larger arc of using this for our own services (Mediawiki, RESTBase, etc.), potentially in production?
Hi Brian,
I think we said this somewhere before, but yes we will consider if kubernetes is a viable platform to run services in production onto.
But Kubernetes is much more than just "containers", it is a distributed computing environment directly derived by Google's own Borg system. I think it's a good candidate to be a platform to run some services onto, but probably not mediawiki or Restbase for the time being.
Cheers,
Giuseppe
wikitech-l@lists.wikimedia.org