In general, if you are using a Python virtual environment on k8s, you
should only modify it from a shell in a matching k8s container. The
easiest way to do this is with `webservice --backend=kubernetes
python3.9 shell` (or whatever version you are using). If the python
version you're using matches what's in use on the Buster bastion, it
should work, but using a k8s shell will be the most reliable.
On Tue, Apr 5, 2022 at 5:16 AM David Caro <dcaro(a)wikimedia.org> wrote:
>
>
> You can use the one used for k8s on the bastion too, the thing is that the k8s
environment expects it to be in a
> specific path, that is `$HOME/pyenv/`.
>
> Now, that might not work if the python version in the bastion is too different from
the one the k8s job has, so you
> might have to have two different ones in that case yes (bastions are restricted to
the python version of the OS, the k8s
> jobs can use different ones depending on the image).
>
>
> On 04/05 11:08, Martin Urbanec wrote:
> > Hello everyone,
> >
> > Thanks for the suggestion. Does this mean that I have to maintain 2 virtual
> > environments (one for bastion and the other one for k8s pod my code will
> > actually run in)?
> >
> > Martin
> >
> > po 4. 4. 2022 v 10:06 odesílatel Arturo Borrero Gonzalez <
> > aborrero(a)wikimedia.org> napsal:
> >
> > >
> > >
> > > On 4/2/22 14:53, Martin Urbanec wrote:
> > > > Hello,
> > > >
> > > > I just received a dozen emails about grid engine migration. I tried
to
> > > > migrate my personal tool (tool.martin-urbanec) first. This tool
> > > > currently generates a Jupyter-notebook based report daily.
> > > >
> > > > I do that by calling jupyter nbconvert --to html --execute
> > > > community_configuration_usage.ipynb from a virtual environment where
> > > > Jupyter is installed, together with a couple of other Python modules.
> > > >
> > > > I managed to create new virtual environment that works from the new
> > > > Buster bastion, and it works when executed directly from the bastion,
> > > > but I can't get it to execute via the k8s-based engine:
> > >
> > > Your problem may be related to bootstrapping the venv. See if this
> > > information can help you:
> > >
> > >
> > >
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Python#Kubernetes_python…
> > >
> > > This is very similar to what JMC89 replied in the other email.
> > >
> > > --
> > > Arturo Borrero Gonzalez
> > > Site Reliability Engineer
> > > Wikimedia Cloud Services
> > > Wikimedia Foundation
> > >
>
> > _______________________________________________
> > Cloud mailing list -- cloud(a)lists.wikimedia.org
> > List information:
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
>
>
> --
> David Caro
> SRE - Cloud Services
> Wikimedia Foundation <https://wikimediafoundation.org/>
> PGP Signature: 7180 83A2 AC8B 314F B4CE 1171 4071 C7E1 D262 69C3
>
> "Imagine a world in which every single human being can freely share in the
> sum of all knowledge. That's our commitment."
> _______________________________________________
> Cloud mailing list -- cloud(a)lists.wikimedia.org
> List information:
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
