Hi all,
I have long been managing some of my tools using non-interactive
provisioning scripts − historically using Shell [1], and increasingly
moving towards Ansible playbooks [2] [3].
Both methods boil down to:
* SSH onto bastion host
* `become tool`
* Execute steps: git pull, install dependencies,etc.
I have not always been able to fulfill my 'non-interactive' requirement.
For my projects which require Node dependencies, I did have to manually
drop into a shell (webservice --backend=kubernetes nodejs shell) in order
to run npm.
(I also had a bit of a hard time when trying to manage crontabs using
Ansible, as the `crontab` executable override seems to be doing all kinds
of magic − I kind of tinkered until I reached a "looks like it works!"
point ^_^)
As I was reading through the Trusty migration docs [4], it is somewhat
hinted that virtualens (for Python dependencies) should also be installed
in an interactive container shell, and not from the Bastion host [5].
Can someone help me with the following questions?
* Is it appropriate to create python virtual envs from the Bastion host?
* Is there a recommended way to execute inside a kubernetes container
remotely / in a non-interactive fashion (eg using a tool like Ansible)?
* in general, am I doing something fundamentally at odds with the Toolforge
environment with such configuration management?
Thanks!
[1]
https://phabricator.wikimedia.org/diffusion/THER/browse/master/bin/build-py…
[2]
https://phabricator.wikimedia.org/source/tool-wikiloves/browse/master/deplo…
[3]
https://github.com/JeanFred/universalviewer-toolforge/blob/master/deploy/ma…
[4]
https://wikitech.wikimedia.org/wiki/News/Toolforge_Trusty_deprecation#Rebui…
[5] https://phabricator.wikimedia.org/T214086#4890276
--
Jean-Fred
Today we have deployed an updated version of the webservicemonitor
service that we use to help ensure that `webservice
--backend=gridengine ...` processes are actively running on the job
grid. The main change in this new version is that we have implemented
tracking of the timestamp of past restart attempts for each tool and a
restart rate limit. The initial limit we have set for this is 3
restarts per 60 minute sliding window.
This change will not stop a tool maintainer from running `webservice
restart` manually. You can read more of the reasoning behind the
change at <https://phabricator.wikimedia.org/T107878>.
Bryan
--
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Manager, Technical Engagement Boise, ID USA
irc: bd808 v:415.839.6885 x6855
_______________________________________________
Wikimedia Cloud Services announce mailing list
Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud-announce
Reminder: Technical Advice IRC meeting this week **(Wednesday) 4-5 pm UTC**
on #wikimedia-tech.
Question can be asked in English & Persian.
The Technical Advice IRC Meeting is a weekly support event for volunteer
developers. Every Wednesday, two full-time developers are available to help
you with all your questions about Mediawiki, gadgets, tools and more! This
can be anything from "how to get started" over "who would be the best
contact for X" to specific questions on your project.
If you know already what you would like to discuss or ask, please add your
topic to the next meeting:
https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
Hope to see you there!
Michi (for the Technical Advice IRC Meeting crew)
--
Michael F. Schönitzer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
https://wikimedia.de
Unsere Vision ist eine Welt, in der alle Menschen am Wissens der Menschheit
teilhaben, es nutzen und mehren können. Helfen Sie uns dabei!
https://spenden.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
Per our announcement on 2019-01-11 [0], the Ubuntu Trusty job grid is
deprecated and jobs running there need to be manually moved to either
the replacement Debian Stretch job grid or the Toolforge Kubernetes
cluster.
Starting today (2019-02-07) the maintainers of tools that have been
seen to run jobs on the Trusty grid will receive email notices with
the subject "[Toolforge] Tools you maintain are running on Trusty job
grid". These reminders will go out each Thursday through 2019-03-07.
At that point we will increase the frequency of reminders to daily
until we reach the Trusty job grid shutdown date in the last week of
March.
Please see the Toolforge Trusty deprecation news page on wikitech [1]
for more information on how to migrate your grid jobs and things to
watch out for in the process.
== What is changing? ==
* New job grid running Son of Grid Engine on Debian Stretch instances
* New limits on concurrent job execution and job submission by a single tool
* New bastion hosts running Debian Stretch with connectivity to the new job grid
* New versions of PHP, Python2, Python3, and other language runtimes
* New versions of various support libraries
== What should I do? ==
The Cloud Services team has created the Toolforge Trusty
deprecation[1] page on wikitech.wikimedia.org to document basic steps
needed to move webservices, cron jobs, and continuous jobs from the
old Trusty grid to the new Stretch grid. That page also provides more
details on the language runtime and library version changes and will
provide answers to common problems people encounter as we find them.
If the answer to your problem isn't on the wiki, ask for help in the
#wikimedia-cloud IRC channel or file a bug in Phabricator[2].
[0]: https://lists.wikimedia.org/pipermail/cloud-announce/2019-January/000122.ht…
[1]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Trusty_deprecation
[2]: https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?title=Stretch…
Thanks,
Bryan (on behalf of the Toolforge administration team)
--
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Manager, Technical Engagement Boise, ID USA
irc: bd808 v:415.839.6885 x6855
_______________________________________________
Wikimedia Cloud Services announce mailing list
Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud-announce
Reminder: Technical Advice IRC meeting this week **(Wednesday) 4-5 pm UTC**
on #wikimedia-tech.
Question can be asked in English & German.
The Technical Advice IRC Meeting is a weekly support event for volunteer
developers. Every Wednesday, two full-time developers are available to help
you with all your questions about Mediawiki, gadgets, tools and more! This
can be anything from "how to get started" over "who would be the best
contact for X" to specific questions on your project.
If you know already what you would like to discuss or ask, please add your
topic to the next meeting:
https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
Hope to see you there!
Michi (for the Technical Advice IRC Meeting crew)
--
Michael F. Schönitzer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
https://wikimedia.de
Unsere Vision ist eine Welt, in der alle Menschen am Wissens der Menschheit
teilhaben, es nutzen und mehren können. Helfen Sie uns dabei!
https://spenden.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
Reminder: Technical Advice IRC meeting **today (Wednesday) 4-5 pm UTC** on
#wikimedia-tech.
Question can be asked in English, German & Persian.
The Technical Advice IRC Meeting is a weekly support event for volunteer
developers. Every Wednesday, two full-time developers are available to help
you with all your questions about Mediawiki, gadgets, tools and more! This
can be anything from "how to get started" over "who would be the best
contact for X" to specific questions on your project.
If you know already what you would like to discuss or ask, please add your
topic to the next meeting:
https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
Hope to see you there!
Michi (for the Technical Advice IRC Meeting crew)
--
Michael F. Schönitzer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
https://wikimedia.de
Unsere Vision ist eine Welt, in der alle Menschen am Wissens der Menschheit
teilhaben, es nutzen und mehren können. Helfen Sie uns dabei!
https://spenden.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
Cross posting for folks who do not also follow the Wikitech-l list.
Some work from the Wikimedia Foundation's search team that may make
finding things with built-in search on Wikitech a bit nicer.
Bryan
---------- Forwarded message ---------
From: Erik Bernhardson <ebernhardson(a)wikimedia.org>
Date: Thu, Jan 24, 2019 at 5:50 PM
Subject: [Wikitech-l] Change in autocomplete behavior on wikitech and mw.org
To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
The default autocomplete profile has been changed to include sub phrase
matching. Essentially this means the autocomplete matching can start at
various points within the page title, rather than only at the very start.
This has shown to be particularly useful on sites with long or nested page
titles like mw.org and wikitech.
This is only the default, if you don't enjoy the new behaviour it can be
changed back to the default profile (or other, stricter profiles) from the
Special:Preferences Search tab.
Related ticket: https://phabricator.wikimedia.org/T212788
Erik B.
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
--
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Manager, Technical Engagement Boise, ID USA
irc: bd808 v:415.839.6885 x6855