July 2019 - Cloud-admin-feed - lists.wikimedia.org

** PROBLEM alert - cloudstore1008/Persistent high iowait is CRITICAL **
by nagios＠icinga1001.wikimedia.org 12 Jul '19

12 Jul '19

Notification Type: PROBLEM Service: Persistent high iowait Host: cloudstore1008 Address: 208.80.155.125 State: CRITICAL Date/Time: Fri Jul 12 14:23:10 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Labstore https://grafana.wikimedia.org/dashboard/db/labs-monitoring Additional Info: 135.2 ge 10

1 0

** PROBLEM alert - labstore1004/Persistent high iowait is CRITICAL **
by nagios＠icinga1001.wikimedia.org 12 Jul '19

12 Jul '19

Notification Type: PROBLEM Service: Persistent high iowait Host: labstore1004 Address: 10.64.37.19 State: CRITICAL Date/Time: Fri Jul 12 14:19:46 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Labstore https://grafana.wikimedia.org/dashboard/db/labs-monitoring Additional Info: 10.97 ge 10

1 0

** PROBLEM alert - labstore1004/Persistent high iowait is CRITICAL **
by nagios＠icinga1001.wikimedia.org 12 Jul '19

12 Jul '19

Notification Type: PROBLEM Service: Persistent high iowait Host: labstore1004 Address: 10.64.37.19 State: CRITICAL Date/Time: Fri Jul 12 14:17:28 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Labstore https://grafana.wikimedia.org/dashboard/db/labs-monitoring Additional Info: 13.83 ge 10

1 0

** RECOVERY alert - cloudcontrol1004/Check the last execution of designate_floating_ip_ptr_records_updater is OK **
by nagios＠icinga1001.wikimedia.org 11 Jul '19

11 Jul '19

Notification Type: RECOVERY Service: Check the last execution of designate_floating_ip_ptr_records_updater Host: cloudcontrol1004 Address: 208.80.154.132 State: OK Date/Time: Thu Jul 11 22:21:58 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Additional Info: OK: Status of the systemd unit designate_floating_ip_ptr_records_updater

1 0

** ACKNOWLEDGEMENT alert - cloudcontrol1004/Check the last execution of designate_floating_ip_ptr_records_updater is CRITICAL **
by nagios＠icinga1001.wikimedia.org 11 Jul '19

11 Jul '19

Notification Type: ACKNOWLEDGEMENT Service: Check the last execution of designate_floating_ip_ptr_records_updater Host: cloudcontrol1004 Address: 208.80.154.132 State: CRITICAL Date/Time: Thu Jul 11 21:52:07 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Additional Info: CRITICAL: Status of the systemd unit designate_floating_ip_ptr_records_updater

1 0

** ACKNOWLEDGEMENT alert - cloudcontrol1004/Check the last execution of designate_floating_ip_ptr_records_updater is CRITICAL **
by nagios＠icinga1001.wikimedia.org 11 Jul '19

11 Jul '19

Notification Type: ACKNOWLEDGEMENT Service: Check the last execution of designate_floating_ip_ptr_records_updater Host: cloudcontrol1004 Address: 208.80.154.132 State: CRITICAL Date/Time: Thu Jul 11 21:51:35 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Additional Info: CRITICAL: Status of the systemd unit designate_floating_ip_ptr_records_updater

1 0

** PROBLEM alert - cloudcontrol1004/Check the last execution of designate_floating_ip_ptr_records_updater is CRITICAL **
by nagios＠icinga1001.wikimedia.org 11 Jul '19

11 Jul '19

Notification Type: PROBLEM Service: Check the last execution of designate_floating_ip_ptr_records_updater Host: cloudcontrol1004 Address: 208.80.154.132 State: CRITICAL Date/Time: Thu Jul 11 21:51:15 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Additional Info: CRITICAL: Status of the systemd unit designate_floating_ip_ptr_records_updater

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: All k8s worker nodes are healthy is CRITICAL **
by nagios＠icinga1001.wikimedia.org 07 Jul '19

07 Jul '19

Notification Type: PROBLEM Service: toolschecker: All k8s worker nodes are healthy Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Sun Jul 7 03:58:31 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/k8s/nodes/ready - 177 bytes in 0.119 second response time

1 0

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: All Flannel etcd nodes are healthy is OK **
by nagios＠icinga1001.wikimedia.org 07 Jul '19

07 Jul '19

Notification Type: RECOVERY Service: toolschecker: All Flannel etcd nodes are healthy Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Sun Jul 7 03:53:15 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 1.496 second response time

1 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: All Flannel etcd nodes are healthy is CRITICAL **
by nagios＠icinga1001.wikimedia.org 07 Jul '19

07 Jul '19

Notification Type: PROBLEM Service: toolschecker: All Flannel etcd nodes are healthy Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Sun Jul 7 03:47:49 UTC 2019 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/etcd/flannel - 522 bytes in 3.026 second response time

1 0

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed July 2019