Notification Type: RECOVERY
Service: Galera haproxy failover
Host: cloudcontrol1004
Address: 208.80.154.132
State: OK
Date/Time: Mon Jun 6 13:31:07 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/HAProxy
Acknowledged by :
Additional Info:
OK check_failover servers up 32 down 0:
Notification Type: ACKNOWLEDGEMENT
Service: Galera haproxy failover
Host: cloudcontrol1004
Address: 208.80.154.132
State: CRITICAL
Date/Time: Mon Jun 6 13:30:26 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/HAProxy
Acknowledged by Andrew Bogott: I just restarted all these... the alert should recover soon
Additional Info:
CRITICAL check_failover servers up 31 down 1:
Notification Type: RECOVERY
Service: Galera haproxy failover
Host: cloudcontrol1005
Address: 208.80.154.85
State: OK
Date/Time: Mon Jun 6 13:30:21 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/HAProxy
Acknowledged by :
Additional Info:
OK check_failover servers up 31 down 0:
Notification Type: RECOVERY
Service: Galera haproxy failover
Host: cloudcontrol1003
Address: 208.80.154.23
State: OK
Date/Time: Mon Jun 6 13:29:59 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/HAProxy
Acknowledged by :
Additional Info:
OK check_failover servers up 32 down 0:
Notification Type: RECOVERY
Service: Check unit status of backup_cinder_volumes
Host: cloudcontrol1005
Address: 208.80.154.85
State: OK
Date/Time: Mon Jun 6 10:37:59 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Check_u…
Acknowledged by :
Additional Info:
OK: Status of the systemd unit backup_cinder_volumes
Notification Type: PROBLEM
Service: Check unit status of backup_cinder_volumes
Host: cloudcontrol1005
Address: 208.80.154.85
State: CRITICAL
Date/Time: Sat Jun 4 21:39:29 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Check_u…
Acknowledged by :
Additional Info:
CRITICAL: Status of the systemd unit backup_cinder_volumes
Notification Type: RECOVERY
Service: Check for VMs leaked by the nova-fullstack test
Host: cloudcontrol1003
Address: 208.80.154.23
State: OK
Date/Time: Sat Jun 4 20:59:45 UTC 2022
Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Check_f…
Acknowledged by :
Additional Info:
3 instances in the admin-monitoring project