December 2018 - Cloud-admin-feed - lists.wikimedia.org

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: Start a job and verify on Trusty is OK **
by nagios＠icinga1001.wikimedia.org 21 Dec '18

21 Dec '18

Notification Type: RECOVERY Service: toolschecker: Start a job and verify on Trusty Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Fri Dec 21 16:21:38 UTC 2018 Notes URLs: Additional Info: HTTP OK: HTTP/1.1 200 OK - 166 bytes in 0.393 second response time

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: Start a job and verify on Trusty is CRITICAL **
by nagios＠icinga1001.wikimedia.org 21 Dec '18

21 Dec '18

Notification Type: PROBLEM Service: toolschecker: Start a job and verify on Trusty Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Fri Dec 21 16:16:34 UTC 2018 Notes URLs: Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/grid/start/trusty - 185 bytes in 9.274 second response time

1 0

** ACKNOWLEDGEMENT alert - labvirt1013/ensure kvm processes are running is CRITICAL **
by nagios＠icinga1001.wikimedia.org 21 Dec '18

21 Dec '18

Notification Type: ACKNOWLEDGEMENT Service: ensure kvm processes are running Host: labvirt1013 Address: 10.64.20.29 State: CRITICAL Date/Time: Fri Dec 21 16:05:58 UTC 2018 Notes URLs: Additional Info: PROCS CRITICAL: 0 processes with regex args /usr/bin/kvm

1 0

** PROBLEM alert - labvirt1013/ensure kvm processes are running is CRITICAL **
by nagios＠icinga1001.wikimedia.org 21 Dec '18

21 Dec '18

Notification Type: PROBLEM Service: ensure kvm processes are running Host: labvirt1013 Address: 10.64.20.29 State: CRITICAL Date/Time: Fri Dec 21 16:04:50 UTC 2018 Notes URLs: Additional Info: PROCS CRITICAL: 0 processes with regex args /usr/bin/kvm

1 0

Host UP alert for wikitech-static.wikimedia.org!
by nagios＠icinga1001.wikimedia.org 13 Dec '18

13 Dec '18

Notification Type: RECOVERY Host: wikitech-static.wikimedia.org State: UP Address: wikitech-static.wikimedia.org Info: PING OK - Packet loss = 0%, RTA = 24.49 ms Date/Time: Thu Dec 13 16:24:18 UTC 2018

1 0

Host DOWN alert for wikitech-static.wikimedia.org!
by nagios＠icinga1001.wikimedia.org 13 Dec '18

13 Dec '18

Notification Type: PROBLEM Host: wikitech-static.wikimedia.org State: DOWN Address: wikitech-static.wikimedia.org Info: CRITICAL - Network Unreachable (wikitech-static.wikimedia.org) Date/Time: Thu Dec 13 16:22:56 UTC 2018

1 0

** RECOVERY alert - labstore1007/High load average is OK **
by nagios＠icinga1001.wikimedia.org 12 Dec '18

12 Dec '18

Notification Type: RECOVERY Service: High load average Host: labstore1007 Address: 208.80.155.106 State: OK Date/Time: Wed Dec 12 12:52:06 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/labs-monitoring Additional Info: OK: Less than 85.00% above the threshold [16.0]

1 0

** PROBLEM alert - labstore1007/High load average is CRITICAL **
by nagios＠icinga1001.wikimedia.org 12 Dec '18

12 Dec '18

Notification Type: PROBLEM Service: High load average Host: labstore1007 Address: 208.80.155.106 State: CRITICAL Date/Time: Wed Dec 12 12:30:38 UTC 2018 Notes URLs: https://grafana.wikimedia.org/dashboard/db/labs-monitoring Additional Info: CRITICAL: 100.00% of data above the critical threshold [24.0]

1 0

** RECOVERY alert - cloudvirt1023/ensure kvm processes are running is OK **
by nagios＠icinga1001.wikimedia.org 10 Dec '18

10 Dec '18

Notification Type: RECOVERY Service: ensure kvm processes are running Host: cloudvirt1023 Address: 10.64.20.42 State: OK Date/Time: Mon Dec 10 23:49:02 UTC 2018 Notes URLs: Additional Info: PROCS OK: 95 processes with regex args qemu-system-x86_64

1 0

** PROBLEM alert - cloudvirt1023/ensure kvm processes are running is CRITICAL **
by nagios＠icinga1001.wikimedia.org 10 Dec '18

10 Dec '18

Notification Type: PROBLEM Service: ensure kvm processes are running Host: cloudvirt1023 Address: 10.64.20.42 State: CRITICAL Date/Time: Mon Dec 10 23:47:50 UTC 2018 Notes URLs: Additional Info: PROCS CRITICAL: 96 processes with regex args qemu-system-x86_64

1 0

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed December 2018