March 2018 - Cloud-admin - lists.wikimedia.org

Re: [Cloud-admin] ** PROBLEM alert - labstore1006/Incoming network saturation is CRITICAL **
by Chase Pettet 30 Mar '18

30 Mar '18

I silenced this as well On Fri, Mar 30, 2018, 3:01 PM <icinga(a)einsteinium.wikimedia.org> wrote: > Notification Type: PROBLEM > > Service: Incoming network saturation > Host: labstore1006 > Address: 208.80.154.7 > State: CRITICAL > > Date/Time: Fri Mar 30 20:01:08 UTC 2018 > > Notes URLs: https://grafana.wikimedia.org/dashboard/db/labs-monitoring > > Additional Info: > > CRITICAL: 22.22% of data above the critical threshold [106250000.0] >

1 0

Re: [Cloud-admin] ** PROBLEM alert - labstore1007/Incoming network saturation is CRITICAL **
by Chase Pettet 30 Mar '18

30 Mar '18

I silenced this system to Sunday as it is not in service but it seems we need to adjust thresholds for when it is? On Fri, Mar 30, 2018, 7:22 AM <icinga(a)einsteinium.wikimedia.org> wrote: > Notification Type: PROBLEM > > Service: Incoming network saturation > Host: labstore1007 > Address: 208.80.155.106 > State: CRITICAL > > Date/Time: Fri Mar 30 12:22:11 UTC 2018 > > Notes URLs: https://grafana.wikimedia.org/dashboard/db/labs-monitoring > > Additional Info: > > CRITICAL: 22.22% of data above the critical threshold [106250000.0] >

2 1

Cron <root@labstore2004> /usr/local/sbin/block_sync 10.64.37.20 misc misc-project misc-snap backup misc-project misc-project-backup 2T
by root＠labstore2004.codfw.wmnet 29 Mar '18

29 Mar '18

2018-03-28 20:00:02,503 INFO force is enabled 2018-03-28 20:00:02,578 INFO removing misc-project-backup 2018-03-28 20:00:02,711 INFO removing misc-project-backup 2018-03-28 20:00:03,268 INFO creating misc-project-backup at 2T 2018-03-28 20:00:04,159 INFO force is enabled 2018-03-28 20:00:04,195 INFO removing misc-snap 2018-03-28 20:00:04,257 INFO removing misc-snap 2018-03-28 20:00:04,805 INFO creating misc-snap at 1T

1 0

Ops meeting/clinic duty update
by Brooke Storm 27 Mar '18

27 Mar '18

Hello all, From the ops meeting yesterday: * Most important part for cloud that the DBAs will be out on the 29th and 30th, so no serious DB changes during that time. Clinic Duty update: * /tmp cleanup on tools-webgrid-lighttpd-1422 - https://phabricator.wikimedia.org/T190185 <https://phabricator.wikimedia.org/T190185> ** Will be adding tmpreaper on web nodes after messing with it a bit. * novaproxy01 filled up with map tiles nginx logs https://phabricator.wikimedia.org/T190218 <https://phabricator.wikimedia.org/T190218> ** Added new settings to logrotate, but I still need to change the cron for logrotate for them to work. * Fixed broken NFS mount on tools-exec-1408 -- Brooke Storm Operations Engineer, Cloud Services

1 0

Reminder I am away :)
by Chase Pettet 22 Mar '18

22 Mar '18

Traveling and wedding family stuff through Sunday. Also, I seem to not be getting emails from phabricator. I don't know why but it must be somewhat recent.

1 0

Cron <root@labstore2004> /usr/local/sbin/block_sync 10.64.37.20 misc misc-project misc-snap backup misc-project misc-project-backup 2T
by root＠labstore2004.codfw.wmnet 22 Mar '18

22 Mar '18

2018-03-21 20:00:03,213 INFO force is enabled 2018-03-21 20:00:03,287 INFO removing misc-project-backup 2018-03-21 20:00:03,420 INFO removing misc-project-backup 2018-03-21 20:00:04,131 INFO creating misc-project-backup at 2T 2018-03-21 20:00:05,027 INFO force is enabled 2018-03-21 20:00:05,055 INFO removing misc-snap 2018-03-21 20:00:05,102 INFO removing misc-snap 2018-03-21 20:00:05,689 INFO creating misc-snap at 1T

1 0

Cron <root@labstore2003> /usr/local/sbin/block_sync 10.64.37.20 tools tools-project tools-snap backup tools-project tools-project-backup 2T
by root＠labstore2003.codfw.wmnet 21 Mar '18

21 Mar '18

2018-03-20 20:00:03,402 INFO force is enabled 2018-03-20 20:00:03,435 INFO removing tools-project-backup 2018-03-20 20:00:03,485 INFO removing tools-project-backup 2018-03-20 20:00:04,045 INFO creating tools-project-backup at 2T 2018-03-20 20:00:04,831 INFO force is enabled 2018-03-20 20:00:04,884 INFO removing tools-snap 2018-03-20 20:00:04,941 INFO removing tools-snap 2018-03-20 20:00:05,958 INFO creating tools-snap at 1T

1 0

[clinic-duty] weekly update
by Arturo Borrero Gonzalez 20 Mar '18

20 Mar '18

Hi there! an update my clinic-duty week: * reboot tools-webgrid-lighttpd-1421, tools-webgrid-lighttpd-1415, tools-webgrid-lighttpd-1420 due to /tmp almost full * reboot tools-exec-1408 due to high load * request creation of WCDO VPS project https://phabricator.wikimedia.org/T189165 approved toolforge access: * fakejames https://toolsadmin.wikimedia.org/tools/membership/status/268 * Jim Smith https://toolsadmin.wikimedia.org/tools/membership/status/269 * Rguptawiki (rgupta) https://toolsadmin.wikimedia.org/tools/membership/status/271 * Ross Mallett (rmallett) https://toolsadmin.wikimedia.org/tools/membership/status/273 * Tigraan (tigraan) https://toolsadmin.wikimedia.org/tools/membership/status/272 declined toolforge access: * https://toolsadmin.wikimedia.org/tools/membership/status/242

1 0

Cron <root@labstore2004> /usr/local/sbin/block_sync 10.64.37.20 misc misc-project misc-snap backup misc-project misc-project-backup 2T
by root＠labstore2004.codfw.wmnet 15 Mar '18

15 Mar '18

2018-03-14 20:00:02,788 INFO force is enabled 2018-03-14 20:00:02,855 INFO removing misc-project-backup 2018-03-14 20:00:02,971 INFO removing misc-project-backup 2018-03-14 20:00:03,716 INFO creating misc-project-backup at 2T 2018-03-14 20:00:04,669 INFO force is enabled 2018-03-14 20:00:04,687 INFO removing misc-snap 2018-03-14 20:00:04,742 INFO removing misc-snap 2018-03-14 20:00:05,178 INFO creating misc-snap at 1T

1 0

Fwd: [Wikimania-l] Wikimania 2018 Call for Submissions
by Bryan Davis 14 Mar '18

14 Mar '18

---------- Forwarded message ---------- From: Liam Wyatt <liamwyatt(a)gmail.com> Date: Sun, Feb 4, 2018 at 11:21 PM Subject: [Wikimania-l] Wikimania 2018 Call for Submissions To: "Wikimania general list (open subscription)" <wikimania-l(a)lists.wikimedia.org>, Wikimedia Mailing List <wikimedia-l(a)lists.wikimedia.org> Cc: Program committee list <wikimania-program(a)lists.wikimedia.org> Dear Wikimedia community, We are pleased to announce that Wikimania 2018 is now accepting proposals for workshops, discussions, presentations, or research posters to give during the conference. To read the full instructions visit the event wiki and click on the link provided there to make your proposal: https://wikimania2018.wikimedia.org/wiki/Submissions The deadline for submissions is 23:59 UTC on Sunday March 18, 2018. This is approximately 6 weeks away. Whether you are a community member of one of the Wikimedia projects, or a fellow open content creator or consumer, we welcome your proposal for a session. Theme This year, the conference will be taking place in Cape Town, South Africa, where the organisers are giving this Wikimania a unique flavor — an explicit theme based in African philosophy: “Bridging knowledge gaps, the ubuntu way forward.” Read more about this theme, why it was chosen, and what it means for the conference program at the Wikimedia blog: https://blog.wikimedia.org/2018/02/05/wikimania-cape-town-ubuntu/ Throughout the conference program, this theme will be tightly held, but loosely defined - in order to encourage a diverse range of responses to the theme. It is our hope that this theme will give us the opportunity to further our goal of creating the “sum of human knowledge”, by encouraging greater diversity and inclusion in who participates and what we discuss at Wikimania. To learn more, and to make a proposal for the conference, please visit: https://wikimania2018.wikimedia.org/wiki/Submissions Please forward this announcement to other lists and groups across the Wikimedia movement. We look forward to reading your submissions. Sincerely, Program committee co-chairs Emna Mizouni, Felix Nartey, and Liam Wyatt. _______________________________________________ Wikimania-l mailing list Wikimania-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikimania-l -- Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org> [[m:User:BDavis_(WMF)]] Manager, Cloud Services Boise, ID USA irc: bd808 v:415.839.6885 x6855

2 2

2024

2023

2022

2021

2020

2019

2018

2017

Cloud-admin March 2018