Notification Type: RECOVERY
Service: DPKG
Host: cloudvirt1022
Address: 10.64.20.41
State: OK
Date/Time: Mon Oct 22 21:28:27 UTC 2018
Notes URLs:
Additional Info:
All packages OK
Notification Type: RECOVERY
Service: High load average
Host: labstore1006
Address: 208.80.154.7
State: OK
Date/Time: Fri Oct 19 16:55:24 UTC 2018
Notes URLs: https://grafana.wikimedia.org/dashboard/db/labs-monitoring
Additional Info:
OK: Less than 85.00% above the threshold [16.0]
Notification Type: PROBLEM
Service: High load average
Host: labstore1006
Address: 208.80.154.7
State: CRITICAL
Date/Time: Fri Oct 19 16:29:54 UTC 2018
Notes URLs: https://grafana.wikimedia.org/dashboard/db/labs-monitoring
Additional Info:
CRITICAL: 100.00% of data above the critical threshold [24.0]
Notification Type: RECOVERY
Service: puppet last run
Host: labvirt1010
Address: 10.64.20.26
State: OK
Date/Time: Fri Oct 19 06:59:43 UTC 2018
Notes URLs:
Additional Info:
OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures
Notification Type: PROBLEM
Service: puppet last run
Host: labvirt1010
Address: 10.64.20.26
State: CRITICAL
Date/Time: Fri Oct 19 06:29:04 UTC 2018
Notes URLs:
Additional Info:
CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/apparmor.d/abstractions/ssl_certs]
Notification Type: RECOVERY
Service: toolschecker: All k8s worker nodes are healthy
Host: checker.tools.wmflabs.org
Address: checker.tools.wmflabs.org
State: OK
Date/Time: Fri Oct 19 04:37:58 UTC 2018
Notes URLs:
Additional Info:
HTTP OK: HTTP/1.1 200 OK - 166 bytes in 0.163 second response time
Notification Type: ACKNOWLEDGEMENT
Service: toolschecker: All k8s worker nodes are healthy
Host: checker.tools.wmflabs.org
Address: checker.tools.wmflabs.org
State: CRITICAL
Date/Time: Fri Oct 19 03:59:39 UTC 2018
Notes URLs:
Additional Info:
HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/k8s/nodes/ready - 185 bytes in 0.200 second response time