Hello all,
Some tools running on the Toolforge Kubernetes cluster are currently
suffering from network failures. It's not yet fully diagnosed, although
we have some ideas as to how to at least reduce the impact. The
tracking bug is https://phabricator.wikimedia.org/T182722.
We'll send another update when we have more information and/or when
things are resolved; in the meantime no action is required on your part
as we'll most likely restart affected tools and services ourselves as
part of fixing the problem.
Sorry for the downtime!
-Andrew + the WMCS team
_______________________________________________
Wikimedia Cloud Services announce mailing list
Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud-announce
Sorry for cross-posting!
Reminder: Technical Advice IRC meeting again **tomorrow, Wednesday 4-5 pm
UTC** on #wikimedia-tech.
The Technical Advice IRC meeting is open for all volunteer developers,
topics and questions. This can be anything from "how to get started" over
"who would be the best contact for X" to specific questions on your project.
If you know already what you would like to discuss or ask, please add your
topic to the next meeting: https://www.mediawiki.org/wiki/Technical_Advice_
IRC_Meeting
**Please note:** We will do a two week break due to the holiday season!
This will be the last Meeting for 2017, the next meeting after this will be
on January 3rd 2018!
Hope to see you there!
Michi (for WMDE’s tech team)
--
Michael F. Schönitzer
Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de
Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
Sorry for cross-posting!
Reminder: Technical Advice IRC meeting again **today, Wednesday 4-5 pm
UTC** on #wikimedia-tech.
The Technical Advice IRC meeting is open for all volunteer developers,
topics and questions. This can be anything from "how to get started" over
"who would be the best contact for X" to specific questions on your project.
If you know already what you would like to discuss or ask, please add your
topic to the next meeting: https://www.mediawiki.org/wiki/Technical_
Advice_IRC_Meeting
Hope to see you there!
Michi (for WMDE’s tech team)
--
Michael F. Schönitzer
Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de
Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
If you are looking for help on irc, phabricator, or this mailing list
and The Cloud Services team folks are slower to respond than usual,
the reason is that we are all together in Austin for 2 days of
meetings as a team and another 3 days of attending
<http://events.linuxfoundation.org/events/kubecon-and-cloudnativecon-north-a…>.
We hope to come back with some ideas about what major projects we will
be working on next and with lots of new information about Kubernetes
and other software that we are using in Toolforge and the rest of
Cloud VPS. Look for updates on our blog
(<https://phabricator.wikimedia.org/phame/blog/view/5/>) in the coming
weeks.
Bryan
--
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Manager, Cloud Services Boise, ID USA
irc: bd808 v:415.839.6885 x6855
A series of upgrades and changes have left instances with
'role::puppetmaster::standalone' applied in a broken state. This is
unfortunate because Puppet is unable to fix itself. There is a small
manual update required.
On the project specific puppet master (instance with
role::puppetmaster:;standalone applied):
0. Make sure your copy of the operation puppet repo is current at
/var/lib/git/operations/puppet
1. edit /etc/puppet/puppet.conf to replace 'default_manifest =
$confdir/manifests/site.pp' with 'default_manifest = $confdir/manifests/'
2. restart apache (service apache2 restart)
3. puppet agent --test
Puppet should now able to fix itself and clients should run fine. Places
where this role is applied can be found at
https://tools.wmflabs.org/openstack-browser/puppetclass/role::puppetmaster:…
--
Chase Pettet
chasemp on phabricator <https://phabricator.wikimedia.org/p/chasemp/> and
IRC
There is a bit of overload [0] on labsdb1003 right now; that is normal
considering there is only 1 server left of the original 3. However, it is
now a good opportunity to start using the new servers [1] so your queries
can be 10x faster!
Another tip is, if you have regularly executed queries (for example, in a
cron), to setup some locking mechanism so one makes sure the same query is
not running multiple times on at the same time (e.g. if for some reasons
they become slower than usual).
Please ask questions if you have them about how to do any of these.
Regards,
[0] <url:
https://grafana.wikimedia.org/dashboard/file/server-board.json?panelId=12&f…
>
[1] <url:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Connecting_to_t…
>
--
Jaime Crespo
<http://wikimedia.org>
Sorry for cross-posting!
Reminder: Technical Advice IRC meeting again **tomorrow, Wednesday 4-5 pm
UTC** on #wikimedia-tech.
The Technical Advice IRC meeting is open for all volunteer developers,
topics and questions. This can be anything from "how to get started" over
"who would be the best contact for X" to specific questions on your project.
If you know already what you would like to discuss or ask, please add your
topic to the next meeting: https://www.mediawiki.org/wiki
/Technical_Advice_IRC_Meeting
This meeting is an offer by WMDE’s tech team. Hosts of tomorrows meeting
are: @Thiemo_WMDE & @Amir1.
Hope to see you there!
Michi (for WMDE’s tech team)
--
Michael F. Schönitzer
Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de
Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
Sorry for cross-posting!
Reminder: Technical Advice IRC meeting again **tomorrow, Wednesday 4-5 pm
UTC** on #wikimedia-tech.
The Technical Advice IRC meeting is open for all volunteer developers,
topics and questions. This can be anything from "how to get started" over
"who would be the best contact for X" to specific questions on your project.
If you know already what you would like to discuss or ask, please add your
topic to the next meeting: https://www.mediawiki.org/
wiki/Technical_Advice_IRC_Meeting
This meeting is an offer by WMDE’s tech team. Hosts of tomorrows meeting
are: @Lucas_WMDE & @Thiemo_WMDE.
Hope to see you there!
Michi (for WMDE’s tech team)
--
Michael F. Schönitzer
Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de
Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
As discussed previously in this list [1] and on phabricator [2], I've
just removed the Ubuntu Trusty image as a default option when creating
new VMs. This is part of a longterm foundation-wide process to
standardize on Debian as the distribution of choice.
Existing Trusty VMs are unaffected by this change, as are present
ToolForge workflows. WMCS operators still have the ability to create
Trusty VMs in a pinch, so if you need one please create a phabricator
task with an explanation of what you need and why and we'll create it as
soon as we're able.
-Andrew
[1] https://lists.wikimedia.org/pipermail/cloud/2017-October/000056.html
[2] https://phabricator.wikimedia.org/T161899
_______________________________________________
Wikimedia Cloud Services announce mailing list
Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud-announce
Sorry for cross-posting!
Reminder: Technical Advice IRC meeting again **tomorrow, Wednesday 4-5 pm
UTC** on #wikimedia-tech.
The Technical Advice IRC meeting is open for all volunteer developers,
topics and questions. This can be anything from "how to get started" over
"who would be the best contact for X" to specific questions on your project.
If you know already what you would like to discuss or ask, please add your
topic to the next meeting: https://www.mediawiki.org/wiki/Technical_
Advice_IRC_Meeting
This meeting is an offer by WMDE’s tech team. Hosts of tomorrows meeting
are: @addshore & @CFisch_WMDE.
Hope to see you there!
Michi (for WMDE’s tech team)
--
Michael F. Schönitzer
Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de
Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.