<div dir="ltr"><div><div><div><div>Been thinking about this a bit and here is my take atm. Numbers reference Yuvi's original outline here. Initial thought is do 1, then look at 4 as the best thing, because we will end up recreating a lot of what 3 provides in the service of doing 2. 3 makes the most sense once 4 and 2 are resolved.<br><br></div>I have poked at and seen some in-depth demos of openshift and it explicitly does not wall off direct kubectl use, and I don't think anything ever will. It may be that there are security contexts in which the really PaaS-y services are boxed in and restricted in certain ways but I'm not worried about an overlay mucking things up too much for now. What I've seen is more straightforward and the productization is very modular in a typical k8s way. I could be proven wrong but so far that's the trend I understand. That being said. I want to push forward and have the migration of webservices be #1. I think that use case is distinct enough to warrant buttoning it up before we move on to straight adhoc or general jobs. Do we have some idea of how many webservices spawn jobs on the grid? is it like 5 or 100? At the moment it's hard to reason on that part of this without knowing how much of an outlier.<br><br></div>I think another thing is mostly necessary to say too: no-ux change is not possible. <br><br></div>It doesn't matter how neatly we tie the bow on this the illusion will fade away the moment ancillary things are different. No way should we rewrite qstat etc to make believe k8s isn't the backend, and I think that jstart/jsub are just the beginning of this path. So I'm in favor of really opinioned change overs that come as full scale changes in expectation on our part and we present the conceptual uses as if it were a public API. We don't really announce or make any effort to proselytize until we feel good about it and we don't muddy the waters with hand wringing over users who only ever wanted to see output in one certain format. I'm both sensitive to the idea that our users are especially burdened by changes and I feel like it does us no good to placate them. Some amount of these changes are going to be the paternalistic "for your own good" type of things and I think that's necessary and as long as it's coupled with responsible release and versioning of the /experience/ we are going to be ok. We won't be in this place until 4 is nailed down (either way) even if that's painful and hard to figure out how we are going to do it.<br><br></div>There is a lot more to say I guess but that's where I'm at for the moment :)<br><div><div><div><div><div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 6, 2016 at 4:18 PM, Yuvi Panda <span dir="ltr"><<a href="mailto:yuvipanda@gmail.com" target="_blank">yuvipanda@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">So I worked a bit with Krinkle (who knows how to use kubectl already)<br>
on getting one of his new bots setup on k8s.<br>
<br>
Conclusions:<br>
Plain kubectl is no-go for most regular users. To mount NFS (and the<br>
nslcd socket, for passwd / group dbs), we *need* to write a custom<br>
yaml file.<br>
<br>
Other than that, it was fairly smooth. But this does mean we need to<br>
wrap kubectl run at least in some form. We could make that super<br>
minimal, but it's modifications from upstream either way.<br>
<br>
Other stuff (kubectl get pods, kubectl delete, logs, etc) are fine.<br>
<div class="gmail-HOEnZb"><div class="gmail-h5"><br>
______________________________<wbr>_________________<br>
Labs-admin mailing list<br>
<a href="mailto:Labs-admin@lists.wikimedia.org">Labs-admin@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-admin" rel="noreferrer" target="_blank">https://lists.wikimedia.org/<wbr>mailman/listinfo/labs-admin</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>Chase Pettet<br></div>Engineering Manager -- Labs<br></div><div>chasemp on <a href="https://phabricator.wikimedia.org/p/chasemp/" target="_blank">phabricator</a> and IRC<br></div></div></div></div></div></div>
</div></div></div></div></div></div></div></div></div>