<div dir="ltr"><div class="gmail_default" style="font-family:tahoma,sans-serif">What does "webservice" refer to here? My labs projects are not serving the web (they are mostly bots); would that mean I won't benefit from k8s?<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Nov 6, 2016 at 1:07 AM, Martin Urbanec <span dir="ltr"><<a href="mailto:martin.urbanec@wikimedia.cz" target="_blank">martin.urbanec@wikimedia.cz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">You can migrate yourself by running webservice --backend=gridengine stop && webservice --backend=kubernetes start from the default to kubernetes and webservice --backend=kubernetes stop && webservice --backend=gridengine start for switching from kubernetes to the default and then try it. You can run it any time you want. <div><br></div><div>Martin</div></div><br><div class="gmail_quote"><div dir="ltr">ne 6. 11. 2016 v 1:11 odesílatel Huji Lee <<a href="mailto:huji.huji@gmail.com" target="_blank">huji.huji@gmail.com</a>> napsal:<br></div><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="m_166037925208104417gmail_msg"><div class="gmail_default m_166037925208104417gmail_msg" style="font-family:tahoma,sans-serif">"Kubernetes" was the word I couldn't remember.<br class="m_166037925208104417gmail_msg"><br class="m_166037925208104417gmail_msg"></div><div class="gmail_default m_166037925208104417gmail_msg" style="font-family:tahoma,sans-serif">How can I get a project hosted in k8s? If possible, I don't want my project to be "moved" to k8s; rather, I want a new project to be made in k8s, so I can test it and figure out. If I decide to migrate, then I will ask my actual project to be moved to k8s. Is that possible?<br class="m_166037925208104417gmail_msg"></div></div><div class="gmail_extra m_166037925208104417gmail_msg"><br class="m_166037925208104417gmail_msg"><div class="gmail_quote m_166037925208104417gmail_msg">On Sat, Nov 5, 2016 at 2:17 PM, Bryan Davis <span dir="ltr" class="m_166037925208104417gmail_msg"><<a href="mailto:bd808@wikimedia.org" class="m_166037925208104417gmail_msg" target="_blank">bd808@wikimedia.org</a>></span> wrote:<br class="m_166037925208104417gmail_msg"><blockquote class="gmail_quote m_166037925208104417gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="m_166037925208104417gmail_msg">On Sat, Nov 5, 2016 at 8:37 AM, Huji Lee <<a href="mailto:huji.huji@gmail.com" class="m_166037925208104417gmail_msg" target="_blank">huji.huji@gmail.com</a>> wrote:<br class="m_166037925208104417gmail_msg">
> Yeah I think it has to do with NFS too. The slow part is clearly the I/O<br class="m_166037925208104417gmail_msg">
> operations.<br class="m_166037925208104417gmail_msg">
><br class="m_166037925208104417gmail_msg">
> I heard there is a new container we are working on in Labs (whose name I<br class="m_166037925208104417gmail_msg">
> forgot) which is faster that NFS. Is that virtualenv?<br class="m_166037925208104417gmail_msg">
<br class="m_166037925208104417gmail_msg">
</span>Webservices running with the kubernetes backend [0] have demonstrated<br class="m_166037925208104417gmail_msg">
much faster NFS access for things like Python virtualenvs. This is<br class="m_166037925208104417gmail_msg">
because the kubernetes containers (called "pods" in the upstream<br class="m_166037925208104417gmail_msg">
documentation) are configured to use an NFS cache layer that is not<br class="m_166037925208104417gmail_msg">
enabled on the OGE grid nodes or the bastions.<br class="m_166037925208104417gmail_msg">
<br class="m_166037925208104417gmail_msg">
The reasons that the grid and bastions do not have this NFS cache<br class="m_166037925208104417gmail_msg">
enabled are a bit murky, but at some point in the past it was the<br class="m_166037925208104417gmail_msg">
cache was disabled as a potential fix for a problem with stale data<br class="m_166037925208104417gmail_msg">
being read by some tools/bots. The techops team is cautious about<br class="m_166037925208104417gmail_msg">
changing the settings to re-enable the cache due lack of understanding<br class="m_166037925208104417gmail_msg">
about what could break as a result. Since the kubernetes environment<br class="m_166037925208104417gmail_msg">
is opt-in and new, they were confident that enabling the cache system<br class="m_166037925208104417gmail_msg">
there would not cause unknown problems.<br class="m_166037925208104417gmail_msg">
<br class="m_166037925208104417gmail_msg">
The larger issue of NFS slowness is to some extent a side effect of<br class="m_166037925208104417gmail_msg">
the growth of Tool Labs usage. More users, tools, and bots reading and<br class="m_166037925208104417gmail_msg">
writing files on the NFS shares cause more contention for the finite<br class="m_166037925208104417gmail_msg">
resources the underlying NFS servers have. A project has been underway<br class="m_166037925208104417gmail_msg">
for several months to migrate the NFS services to a new set of<br class="m_166037925208104417gmail_msg">
servers. This migration is currently scheduled to complete on November<br class="m_166037925208104417gmail_msg">
14th [1]. There are no guarantees that NFS access will be<br class="m_166037925208104417gmail_msg">
significantly faster once the new servers are in use, but the setup<br class="m_166037925208104417gmail_msg">
will be better documented and more familiar to the Labs techops team.<br class="m_166037925208104417gmail_msg">
That should make it easier for them to troubleshoot issues and work<br class="m_166037925208104417gmail_msg">
towards providing a better experience for all of us.<br class="m_166037925208104417gmail_msg">
<br class="m_166037925208104417gmail_msg">
[0]: <a href="https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Web/Kubernetes" rel="noreferrer" class="m_166037925208104417gmail_msg" target="_blank">https://wikitech.wikimedia.<wbr>org/wiki/Help:Tool_Labs/Web/<wbr>Kubernetes</a><br class="m_166037925208104417gmail_msg">
[1]: <a href="https://lists.wikimedia.org/pipermail/labs-announce/2016-November/000177.html" rel="noreferrer" class="m_166037925208104417gmail_msg" target="_blank">https://lists.wikimedia.org/<wbr>pipermail/labs-announce/2016-<wbr>November/000177.html</a><br class="m_166037925208104417gmail_msg">
<span class="m_166037925208104417m_-1504253711175889787HOEnZb m_166037925208104417gmail_msg"><font class="m_166037925208104417gmail_msg" color="#888888"><br class="m_166037925208104417gmail_msg">
Bryan<br class="m_166037925208104417gmail_msg">
--<br class="m_166037925208104417gmail_msg">
Bryan Davis Wikimedia Foundation <<a href="mailto:bd808@wikimedia.org" class="m_166037925208104417gmail_msg" target="_blank">bd808@wikimedia.org</a>><br class="m_166037925208104417gmail_msg">
[[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA<br class="m_166037925208104417gmail_msg">
irc: bd808 v:<a href="tel:415.839.6885%20x6855" value="+14158396885" class="m_166037925208104417gmail_msg" target="_blank">415.839.6885 x6855</a><br class="m_166037925208104417gmail_msg">
</font></span><div class="m_166037925208104417m_-1504253711175889787HOEnZb m_166037925208104417gmail_msg"><div class="m_166037925208104417m_-1504253711175889787h5 m_166037925208104417gmail_msg"><br class="m_166037925208104417gmail_msg">
______________________________<wbr>_________________<br class="m_166037925208104417gmail_msg">
Labs-l mailing list<br class="m_166037925208104417gmail_msg">
<a href="mailto:Labs-l@lists.wikimedia.org" class="m_166037925208104417gmail_msg" target="_blank">Labs-l@lists.wikimedia.org</a><br class="m_166037925208104417gmail_msg">
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" rel="noreferrer" class="m_166037925208104417gmail_msg" target="_blank">https://lists.wikimedia.org/<wbr>mailman/listinfo/labs-l</a><br class="m_166037925208104417gmail_msg">
</div></div></blockquote></div><br class="m_166037925208104417gmail_msg"></div>
______________________________<wbr>_________________<br class="m_166037925208104417gmail_msg">
Labs-l mailing list<br class="m_166037925208104417gmail_msg">
<a href="mailto:Labs-l@lists.wikimedia.org" class="m_166037925208104417gmail_msg" target="_blank">Labs-l@lists.wikimedia.org</a><br class="m_166037925208104417gmail_msg">
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" rel="noreferrer" class="m_166037925208104417gmail_msg" target="_blank">https://lists.wikimedia.org/<wbr>mailman/listinfo/labs-l</a><br class="m_166037925208104417gmail_msg">
</blockquote></div></div></div>
<br>______________________________<wbr>_________________<br>
Labs-l mailing list<br>
<a href="mailto:Labs-l@lists.wikimedia.org">Labs-l@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" rel="noreferrer" target="_blank">https://lists.wikimedia.org/<wbr>mailman/listinfo/labs-l</a><br>
<br></blockquote></div><br></div>